Detecting Spells in Fantasy Literature with a Transformer Based
Artificial Intelligence
- URL: http://arxiv.org/abs/2308.03660v1
- Date: Mon, 7 Aug 2023 15:20:20 GMT
- Title: Detecting Spells in Fantasy Literature with a Transformer Based
Artificial Intelligence
- Authors: Marcel Moravek, Alexander Zender, Andreas M\"uller
- Abstract summary: We use BERT for context-based phrase recognition of magic spells in the Harry Potter novel series.
A pre-trained BERT model was used and fine-tuned utilising different datasets and training methods to identify the searched context.
- Score: 69.85273194899884
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Transformer architectures and models have made significant progress in
language-based tasks. In this area, is BERT one of the most widely used and
freely available transformer architecture. In our work, we use BERT for
context-based phrase recognition of magic spells in the Harry Potter novel
series. Spells are a common part of active magic in fantasy novels. Typically,
spells are used in a specific context to achieve a supernatural effect. A
series of investigations were conducted to see if a Transformer architecture
could recognize such phrases based on their context in the Harry Potter saga.
For our studies a pre-trained BERT model was used and fine-tuned utilising
different datasets and training methods to identify the searched context. By
considering different approaches for sequence classification as well as token
classification, it is shown that the context of spells can be recognised.
According to our investigations, the examined sequence length for fine-tuning
and validation of the model plays a significant role in context recognition.
Based on this, we have investigated whether spells have overarching properties
that allow a transfer of the neural network models to other fantasy universes
as well. The application of our model showed promising results and is worth to
be deepened in subsequent studies.
Related papers
- Context Disentangling and Prototype Inheriting for Robust Visual
Grounding [56.63007386345772]
Visual grounding (VG) aims to locate a specific target in an image based on a given language query.
We propose a novel framework with context disentangling and prototype inheriting for robust visual grounding to handle both scenes.
Our method outperforms the state-of-the-art methods in both scenarios.
arXiv Detail & Related papers (2023-12-19T09:03:53Z) - A Comprehensive Survey on Applications of Transformers for Deep Learning
Tasks [60.38369406877899]
Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data.
transformer models excel in handling long dependencies between input sequence elements and enable parallel processing.
Our survey encompasses the identification of the top five application domains for transformer-based models.
arXiv Detail & Related papers (2023-06-11T23:13:51Z) - Improving Attention-Based Interpretability of Text Classification
Transformers [7.027858121801477]
We study the effectiveness of attention-based interpretability techniques for transformers in text classification.
We show that, with proper setup, attention may be used in such tasks with results comparable to state-of-the-art techniques.
arXiv Detail & Related papers (2022-09-22T09:19:22Z) - Disentangled Action Recognition with Knowledge Bases [77.77482846456478]
We aim to improve the generalization ability of the compositional action recognition model to novel verbs or novel nouns.
Previous work utilizes verb-noun compositional action nodes in the knowledge graph, making it inefficient to scale.
We propose our approach: Disentangled Action Recognition with Knowledge-bases (DARK), which leverages the inherent compositionality of actions.
arXiv Detail & Related papers (2022-07-04T20:19:13Z) - Learning Multiscale Transformer Models for Sequence Generation [33.73729074207944]
We build a multiscale Transformer model by establishing relationships among scales based on word-boundary information and phrase-level prior knowledge.
Notably, it yielded consistent performance gains over the strong baseline on several test sets without sacrificing the efficiency.
arXiv Detail & Related papers (2022-06-19T07:28:54Z) - Multilingual Extraction and Categorization of Lexical Collocations with
Graph-aware Transformers [86.64972552583941]
We put forward a sequence tagging BERT-based model enhanced with a graph-aware transformer architecture, which we evaluate on the task of collocation recognition in context.
Our results suggest that explicitly encoding syntactic dependencies in the model architecture is helpful, and provide insights on differences in collocation typification in English, Spanish and French.
arXiv Detail & Related papers (2022-05-23T16:47:37Z) - Are Neural Language Models Good Plagiarists? A Benchmark for Neural
Paraphrase Detection [5.847824494580938]
We propose a benchmark consisting of paraphrased articles using recent language models relying on the Transformer architecture.
Our contribution fosters future research of paraphrase detection systems as it offers a large collection of aligned original and paraphrased documents.
arXiv Detail & Related papers (2021-03-23T11:01:35Z) - Pretrained Transformers for Text Ranking: BERT and Beyond [53.83210899683987]
This survey provides an overview of text ranking with neural network architectures known as transformers.
The combination of transformers and self-supervised pretraining has been responsible for a paradigm shift in natural language processing.
arXiv Detail & Related papers (2020-10-13T15:20:32Z) - Pay Attention to What You Read: Non-recurrent Handwritten Text-Line
Recognition [4.301658883577544]
We introduce a non-recurrent approach to recognize handwritten text by the use of transformer models.
We are able to tackle character recognition as well as to learn language-related dependencies of the character sequences to be decoded.
arXiv Detail & Related papers (2020-05-26T21:15:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.