Classifying Unreliable Narrators with Large Language Models

• URL: http://arxiv.org/abs/2506.10231v1
• Date: Wed, 11 Jun 2025 23:17:12 GMT
• Title: Classifying Unreliable Narrators with Large Language Models
• Authors: Anneliese Brei, Katharine Henry, Abhisheik Sharma, Shashank Srivastava, Snigdha Chaturvedi,
• Abstract summary: We present TUNa, a human-annotated dataset of narratives from multiple domains.<n>We define classification tasks for intra-narrational, inter-narrational, and inter-textual unreliabilities.<n>We propose learning from literature to perform unreliable narrator classification on real-world text data.
• Score: 23.817691955577835
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Often when we interact with a first-person account of events, we consider whether or not the narrator, the primary speaker of the text, is reliable. In this paper, we propose using computational methods to identify unreliable narrators, i.e. those who unintentionally misrepresent information. Borrowing literary theory from narratology to define different types of unreliable narrators based on a variety of textual phenomena, we present TUNa, a human-annotated dataset of narratives from multiple domains, including blog posts, subreddit posts, hotel reviews, and works of literature. We define classification tasks for intra-narrational, inter-narrational, and inter-textual unreliabilities and analyze the performance of popular open-weight and proprietary LLMs for each. We propose learning from literature to perform unreliable narrator classification on real-world text data. To this end, we experiment with few-shot, fine-tuning, and curriculum learning settings. Our results show that this task is very challenging, and there is potential for using LLMs to identify unreliable narrators. We release our expert-annotated dataset and code and invite future research in this area.

Related papers

• Collaborative Storytelling and LLM: A Linguistic Analysis of Automatically-Generated Role-Playing Game Sessions [55.2480439325792]
  Role-playing games (RPG) are games in which players interact with one another to create narratives.<n>This emerging form of shared narrative, primarily oral, is receiving increasing attention.<n>In this paper, we aim to discover to what extent the language of Large Language Models (LLMs) exhibit oral or written features when asked to generate an RPG session.
  arXiv  Detail & Related papers  (2025-03-26T15:10:47Z)
• Show, Don't Tell: Uncovering Implicit Character Portrayal using LLMs [19.829683714192615]
  We introduce LIIPA, a framework for prompting large language models to uncover implicit character portrayals.<n>We find that LIIPA outperforms existing approaches, and is more robust to increasing character counts.<n>Our work demonstrates the potential benefits of using LLMs to analyze complex characters.
  arXiv  Detail & Related papers  (2024-12-05T19:46:53Z)
• BookWorm: A Dataset for Character Description and Analysis [59.186325346763184]
  We define two tasks: character description, which generates a brief factual profile, and character analysis, which offers an in-depth interpretation. We introduce the BookWorm dataset, pairing books from the Gutenberg Project with human-written descriptions and analyses. Our findings show that retrieval-based approaches outperform hierarchical ones in both tasks.
  arXiv  Detail & Related papers  (2024-10-14T10:55:58Z)
• Mapping News Narratives Using LLMs and Narrative-Structured Text Embeddings [0.0]
  We introduce a numerical narrative representation grounded in structuralist linguistic theory. We extract the actants using an open-source LLM and integrate them into a Narrative-Structured Text Embedding. We demonstrate the analytical insights of the method on the example of 5000 full-text news articles from Al Jazeera and The Washington Post on the Israel-Palestine conflict.
  arXiv  Detail & Related papers  (2024-09-10T14:15:30Z)
• Are Large Language Models Capable of Generating Human-Level Narratives? [114.34140090869175]
  This paper investigates the capability of LLMs in storytelling, focusing on narrative development and plot progression. We introduce a novel computational framework to analyze narratives through three discourse-level aspects. We show that explicit integration of discourse features can enhance storytelling, as is demonstrated by over 40% improvement in neural storytelling.
  arXiv  Detail & Related papers  (2024-07-18T08:02:49Z)
• Improving Quotation Attribution with Fictional Character Embeddings [11.259583037191772]
  We propose to augment a popular quotation attribution system, BookNLP, with character embeddings that encode global stylistic information of characters. We show that combining BookNLP's contextual information with our proposed global character embeddings improves the identification of speakers for anaphoric and implicit quotes.
  arXiv  Detail & Related papers  (2024-06-17T09:46:35Z)
• Detecting Narrative Elements in Informational Text [0.0]
  We introduce NEAT (Narrative Elements AnnoTation) - a novel NLP task for detecting narrative elements in raw text. We use this scheme to annotate a new dataset of 2,209 sentences, compiled from 46 news articles from various category domains. We trained a number of supervised models in several different setups over the annotated dataset to identify the different narrative elements, achieving an average F1 score of up to 0.77.
  arXiv  Detail & Related papers  (2022-10-06T16:23:33Z)
• NECE: Narrative Event Chain Extraction Toolkit [64.89332212585404]
  We introduce NECE, an open-access, document-level toolkit that automatically extracts and aligns narrative events in the temporal order of their occurrence. We show the high quality of the NECE toolkit and demonstrate its downstream application in analyzing narrative bias regarding gender. We also openly discuss the shortcomings of the current approach, and potential of leveraging generative models in future works.
  arXiv  Detail & Related papers  (2022-08-17T04:30:58Z)
• Paragraph-level Commonsense Transformers with Recurrent Memory [77.4133779538797]
  We train a discourse-aware model that incorporates paragraph-level information to generate coherent commonsense inferences from narratives. Our results show that PARA-COMET outperforms the sentence-level baselines, particularly in generating inferences that are both coherent and novel.
  arXiv  Detail & Related papers  (2020-10-04T05:24:12Z)
• Abstractive Summarization of Spoken and Written Instructions with BERT [66.14755043607776]
  We present the first application of the BERTSum model to conversational language. We generate abstractive summaries of narrated instructional videos across a wide variety of topics. We envision this integrated as a feature in intelligent virtual assistants, enabling them to summarize both written and spoken instructional content upon request.
  arXiv  Detail & Related papers  (2020-08-21T20:59:34Z)
• Exploring aspects of similarity between spoken personal narratives by disentangling them into narrative clause types [13.350982138577038]
  We introduce a corpus of real-world spoken personal narratives comprising 10,296 narrative clauses from 594 video transcripts. Second, we ask non-narrative experts to annotate those clauses under Labov's sociolinguistic model of personal narratives. Third, we train a classifier that reaches 84.7% F-score for the highest-agreed clauses. Our approach is intended to help inform machine learning methods aimed at studying or representing personal narratives.
  arXiv  Detail & Related papers  (2020-05-26T14:34:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.