Comparative Analysis of Named Entity Recognition in the Dungeons and Dragons Domain

• URL: http://arxiv.org/abs/2309.17171v1
• Date: Fri, 29 Sep 2023 12:09:36 GMT
• Title: Comparative Analysis of Named Entity Recognition in the Dungeons and Dragons Domain
• Abstract summary: We analyzed 10 Named Entity Recognition (NER) models on 7 Dungeons and Dragons (D&D) adventure books to assess domain-specific performance. Our findings indicate that, without modifications, Flair, Trankit, and Spacy outperform others in identifying named entities in the D&D context.
• Score: 0.3108011671896571
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many NLP tasks, although well-resolved for general English, face challenges in specific domains like fantasy literature. This is evident in Named Entity Recognition (NER), which detects and categorizes entities in text. We analyzed 10 NER models on 7 Dungeons and Dragons (D&D) adventure books to assess domain-specific performance. Using open-source Large Language Models, we annotated named entities in these books and evaluated each model's precision. Our findings indicate that, without modifications, Flair, Trankit, and Spacy outperform others in identifying named entities in the D&D context.

Related papers

• "I've Heard of You!": Generate Spoken Named Entity Recognition Data for Unseen Entities [59.22329574700317]
  Spoken named entity recognition (NER) aims to identify named entities from speech. New named entities appear every day, however, annotating their Spoken NER data is costly. We propose a method for generating Spoken NER data based on a named entity dictionary (NED) to reduce costs.
  arXiv  Detail & Related papers  (2024-12-26T07:43:18Z)
• LFED: A Literary Fiction Evaluation Dataset for Large Language Models [58.85989777743013]
  We collect 95 literary fictions that are either originally written in Chinese or translated into Chinese, covering a wide range of topics across several centuries. We define a question taxonomy with 8 question categories to guide the creation of 1,304 questions. We conduct an in-depth analysis to ascertain how specific attributes of literary fictions (e.g., novel types, character numbers, the year of publication) impact LLM performance in evaluations.
  arXiv  Detail & Related papers  (2024-05-16T15:02:24Z)
• Fine Tuning Named Entity Extraction Models for the Fantasy Domain [0.3108011671896571]
  This work uses available lore of monsters in the D&D domain to fine-tune Trankit, which is a prolific NER framework. The system acquires the ability to extract monster names from relevant domain documents under a novel NER tag.
  arXiv  Detail & Related papers  (2024-02-16T13:11:13Z)
• Multicultural Name Recognition For Previously Unseen Names [65.268245109828]
  This paper attempts to improve recognition of person names, a diverse category that can grow any time someone is born or changes their name. I look at names from 103 countries to compare how well the model performs on names from different cultures. I find that a model with combined character and word input outperforms word-only models and may improve on accuracy compared to classical NER models.
  arXiv  Detail & Related papers  (2024-01-23T17:58:38Z)
• NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval [49.827932299460514]
  We argue that capabilities provided by large language models are not the end of NER research, but rather an exciting beginning. We present three variants of the NER task, together with a dataset to support them. We provide a large, silver-annotated corpus of 4 million paragraphs covering 500 entity types.
  arXiv  Detail & Related papers  (2023-10-22T12:23:00Z)
• Dynamic Named Entity Recognition [5.9401550252715865]
  We introduce a new task: Dynamic Named Entity Recognition (DNER) DNER provides a framework to better evaluate the ability of algorithms to extract entities by exploiting the context. We evaluate baseline models and present experiments reflecting issues and research axes related to this novel task.
  arXiv  Detail & Related papers  (2023-02-16T15:50:02Z)
• Entity Disambiguation with Entity Definitions [50.01142092276296]
  Local models have recently attained astounding performances in Entity Disambiguation (ED) Previous works limited their studies to using, as the textual representation of each candidate, only its Wikipedia title. In this paper, we address this limitation and investigate to what extent more expressive textual representations can mitigate it. We report a new state of the art on 2 out of 6 benchmarks we consider and strongly improve the generalization capability over unseen patterns.
  arXiv  Detail & Related papers  (2022-10-11T17:46:28Z)
• The Fellowship of the Authors: Disambiguating Names from Social Network Context [2.3605348648054454]
  Authority lists with extensive textual descriptions for each entity are lacking and ambiguous named entities. We combine BERT-based mention representations with a variety of graph induction strategies and experiment with supervised and unsupervised cluster inference methods. We find that in-domain language model pretraining can significantly improve mention representations, especially for larger corpora.
  arXiv  Detail & Related papers  (2022-08-31T21:51:55Z)
• Can BERT Dig It? -- Named Entity Recognition for Information Retrieval in the Archaeology Domain [3.928604516640069]
  ArcheoBERTje is a BERT model pre-trained on Dutch archaeological texts. We analyse the differences between the vocabulary and output of the BERT models on the full collection.
  arXiv  Detail & Related papers  (2021-06-14T20:26:19Z)
• Few-NERD: A Few-Shot Named Entity Recognition Dataset [35.669024917327825]
  We present Few-NERD, a large-scale human-annotated few-shot NER dataset with a hierarchy of 8 coarse-grained and 66 fine-grained entity types. Few-NERD consists of 188,238 sentences from Wikipedia, 4,601,160 words are included and each is annotated as context or a part of a two-level entity type.
  arXiv  Detail & Related papers  (2021-05-16T15:53:17Z)
• Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve [49.878051587667244]
  We examine the performance of several variants of LSTM-CRF architectures for named entity recognition. We find that context representations do contribute to system performance, but that the main factor driving high performance is learning the name tokens themselves. We enlist human annotators to evaluate the feasibility of inferring entity types from the context alone and find that, while people are not able to infer the entity type either for the majority of the errors made by the context-only system, there is some room for improvement.
  arXiv  Detail & Related papers  (2020-04-09T14:37:12Z)
• Zero-Resource Cross-Domain Named Entity Recognition [68.83177074227598]
  Existing models for cross-domain named entity recognition rely on numerous unlabeled corpus or labeled NER training data in target domains. We propose a cross-domain NER model that does not use any external resources.
  arXiv  Detail & Related papers  (2020-02-14T09:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.