AmbigDocs: Reasoning across Documents on Different Entities under the Same Name

• URL: http://arxiv.org/abs/2404.12447v3
• Date: Fri, 9 Aug 2024 12:40:51 GMT
• Title: AmbigDocs: Reasoning across Documents on Different Entities under the Same Name
• Abstract summary: Handling confusing entity mentions is a crucial skill for language models (LMs) We introduce a new benchmark, AmbigDocs, to test this ability. We generate questions containing an ambiguous name and their corresponding sets of answers.
• Score: 46.33718186200757
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Different entities with the same name can be difficult to distinguish. Handling confusing entity mentions is a crucial skill for language models (LMs). For example, given the question "Where was Michael Jordan educated?" and a set of documents discussing different people named Michael Jordan, can LMs distinguish entity mentions to generate a cohesive answer to the question? To test this ability, we introduce a new benchmark, AmbigDocs. By leveraging Wikipedia's disambiguation pages, we identify a set of documents, belonging to different entities who share an ambiguous name. From these documents, we generate questions containing an ambiguous name and their corresponding sets of answers. Our analysis reveals that current state-of-the-art models often yield ambiguous answers or incorrectly merge information belonging to different entities. We establish an ontology categorizing four types of incomplete answers and automatic evaluation metrics to identify such categories. We lay the foundation for future work on reasoning across multiple documents with ambiguous entities.

Related papers

• Curate Before You Connect: Identity and Ontology Tagging in a Production Knowledge Graph [0.0]
  This paper describes the ingestion and ontology-tagging layer that turns a validated extraction stream into a knowledge graph.<n>We describe a record-identity ladder that decides sameness from identifier columns, name columns, display names and type-scoped position rather than from name similarity.<n>We quantify the graph's conformance debt, show secondary classifications compensating for a mis-parented primary class, and describe a curation queue grown to 48,403 pending proposals against 775 human decisions.
  arXiv  Detail & Related papers  (2026-08-11T08:30:37Z)
• Generative Retrieval Meets Multi-Graded Relevance [104.75244721442756]
  We introduce a framework called GRaded Generative Retrieval (GR$2$) GR$2$ focuses on two key components: ensuring relevant and distinct identifiers, and implementing multi-graded constrained contrastive training. Experiments on datasets with both multi-graded and binary relevance demonstrate the effectiveness of GR$2$.
  arXiv  Detail & Related papers  (2024-09-27T02:55:53Z)
• Different Tastes of Entities: Investigating Human Label Variation in Named Entity Annotations [23.059491714512077]
  This paper studies disagreements in expert-annotated named entity datasets for three languages: English, Danish, and Bavarian. We show that text ambiguity and artificial guideline changes are dominant factors for diverse annotations among high-quality revisions.
  arXiv  Detail & Related papers  (2024-02-02T14:08:34Z)
• Multicultural Name Recognition For Previously Unseen Names [65.268245109828]
  This paper attempts to improve recognition of person names, a diverse category that can grow any time someone is born or changes their name. I look at names from 103 countries to compare how well the model performs on names from different cultures. I find that a model with combined character and word input outperforms word-only models and may improve on accuracy compared to classical NER models.
  arXiv  Detail & Related papers  (2024-01-23T17:58:38Z)
• The Fellowship of the Authors: Disambiguating Names from Social Network Context [2.3605348648054454]
  Authority lists with extensive textual descriptions for each entity are lacking and ambiguous named entities. We combine BERT-based mention representations with a variety of graph induction strategies and experiment with supervised and unsupervised cluster inference methods. We find that in-domain language model pretraining can significantly improve mention representations, especially for larger corpora.
  arXiv  Detail & Related papers  (2022-08-31T21:51:55Z)
• Parallel Instance Query Network for Named Entity Recognition [73.30174490672647]
  Named entity recognition (NER) is a fundamental task in natural language processing. Recent works treat named entity recognition as a reading comprehension task, constructing type-specific queries manually to extract entities. We propose Parallel Instance Query Network (PIQN), which sets up global and learnable instance queries to extract entities in a parallel manner.
  arXiv  Detail & Related papers  (2022-03-20T13:01:25Z)
• Knowledge-Rich Self-Supervised Entity Linking [58.838404666183656]
  Knowledge-RIch Self-Supervision ($tt KRISSBERT$) is a universal entity linker for four million UMLS entities. Our approach subsumes zero-shot and few-shot methods, and can easily incorporate entity descriptions and gold mention labels if available. Without using any labeled information, our method produces $tt KRISSBERT$, a universal entity linker for four million UMLS entities.
  arXiv  Detail & Related papers  (2021-12-15T05:05:12Z)
• Autoregressive Entity Retrieval [55.38027440347138]
  Entities are at the center of how we represent and aggregate knowledge. The ability to retrieve such entities given a query is fundamental for knowledge-intensive tasks such as entity linking and open-domain question answering. We propose GENRE, the first system that retrieves entities by generating their unique names, left to right, token-by-token in an autoregressive fashion.
  arXiv  Detail & Related papers  (2020-10-02T10:13:31Z)
• Pairwise Multi-Class Document Classification for Semantic Relations between Wikipedia Articles [5.40541521227338]
  We model the problem of finding the relationship between two documents as a pairwise document classification task. To find semantic relation between documents, we apply a series of techniques, such as GloVe, paragraph-s, BERT, and XLNet. We perform our experiments on a newly proposed dataset of 32,168 Wikipedia article pairs and Wikidata properties that define the semantic document relations.
  arXiv  Detail & Related papers  (2020-03-22T12:52:56Z)
• Joint Embedding in Named Entity Linking on Sentence Level [30.229263131244906]
  We propose a new unified embedding method by maximizing the relationships learned from knowledge graphs. We focus on how to link entity for mentions at a sentence level, which reduces the noises introduced by different appearances of the same mention in a document.
  arXiv  Detail & Related papers  (2020-02-12T12:06:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.