Show, Don't Tell: Uncovering Implicit Character Portrayal using LLMs

• URL: http://arxiv.org/abs/2412.04576v1
• Date: Thu, 05 Dec 2024 19:46:53 GMT
• Title: Show, Don't Tell: Uncovering Implicit Character Portrayal using LLMs
• Authors: Brandon Jaipersaud, Zining Zhu, Frank Rudzicz, Elliot Creager,
• Abstract summary: We introduce LIIPA, a framework for prompting large language models to uncover implicit character portrayals.<n>We find that LIIPA outperforms existing approaches, and is more robust to increasing character counts.<n>Our work demonstrates the potential benefits of using LLMs to analyze complex characters.
• Score: 19.829683714192615
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tools for analyzing character portrayal in fiction are valuable for writers and literary scholars in developing and interpreting compelling stories. Existing tools, such as visualization tools for analyzing fictional characters, primarily rely on explicit textual indicators of character attributes. However, portrayal is often implicit, revealed through actions and behaviors rather than explicit statements. We address this gap by leveraging large language models (LLMs) to uncover implicit character portrayals. We start by generating a dataset for this task with greater cross-topic similarity, lexical diversity, and narrative lengths than existing narrative text corpora such as TinyStories and WritingPrompts. We then introduce LIIPA (LLMs for Inferring Implicit Portrayal for Character Analysis), a framework for prompting LLMs to uncover character portrayals. LIIPA can be configured to use various types of intermediate computation (character attribute word lists, chain-of-thought) to infer how fictional characters are portrayed in the source text. We find that LIIPA outperforms existing approaches, and is more robust to increasing character counts (number of unique persons depicted) due to its ability to utilize full narrative context. Lastly, we investigate the sensitivity of portrayal estimates to character demographics, identifying a fairness-accuracy tradeoff among methods in our LIIPA framework -- a phenomenon familiar within the algorithmic fairness literature. Despite this tradeoff, all LIIPA variants consistently outperform non-LLM baselines in both fairness and accuracy. Our work demonstrates the potential benefits of using LLMs to analyze complex characters and to better understand how implicit portrayal biases may manifest in narrative texts.

Related papers

• Collaborative Storytelling and LLM: A Linguistic Analysis of Automatically-Generated Role-Playing Game Sessions [55.2480439325792]
  Role-playing games (RPG) are games in which players interact with one another to create narratives. This emerging form of shared narrative, primarily oral, is receiving increasing attention. In this paper, we aim to discover to what extent the language of Large Language Models (LLMs) exhibit oral or written features when asked to generate an RPG session.
  arXiv  Detail & Related papers  (2025-03-26T15:10:47Z)
• BookWorm: A Dataset for Character Description and Analysis [59.186325346763184]
  We define two tasks: character description, which generates a brief factual profile, and character analysis, which offers an in-depth interpretation. We introduce the BookWorm dataset, pairing books from the Gutenberg Project with human-written descriptions and analyses. Our findings show that retrieval-based approaches outperform hierarchical ones in both tasks.
  arXiv  Detail & Related papers  (2024-10-14T10:55:58Z)
• Improving Quotation Attribution with Fictional Character Embeddings [11.259583037191772]
  We propose to augment a popular quotation attribution system, BookNLP, with character embeddings that encode global stylistic information of characters. We show that combining BookNLP's contextual information with our proposed global character embeddings improves the identification of speakers for anaphoric and implicit quotes.
  arXiv  Detail & Related papers  (2024-06-17T09:46:35Z)
• CHIRON: Rich Character Representations in Long-Form Narratives [98.273323001781]
  We propose CHIRON, a new character sheet' based representation that organizes and filters textual information about characters. We validate CHIRON via the downstream task of masked-character prediction, where our experiments show CHIRON is better and more flexible than comparable summary-based baselines. metrics derived from CHIRON can be used to automatically infer character-centricity in stories, and that these metrics align with human judgments.
  arXiv  Detail & Related papers  (2024-06-14T17:23:57Z)
• LFED: A Literary Fiction Evaluation Dataset for Large Language Models [58.85989777743013]
  We collect 95 literary fictions that are either originally written in Chinese or translated into Chinese, covering a wide range of topics across several centuries. We define a question taxonomy with 8 question categories to guide the creation of 1,304 questions. We conduct an in-depth analysis to ascertain how specific attributes of literary fictions (e.g., novel types, character numbers, the year of publication) impact LLM performance in evaluations.
  arXiv  Detail & Related papers  (2024-05-16T15:02:24Z)
• Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works [33.817319226631426]
  Large language models (LLMs) have demonstrated impressive performance and spurred numerous AI applications. The prerequisite for these RPAs lies in the capability of LLMs to understand characters from fictional works. Previous efforts have evaluated this capability via basic classification tasks or characteristic imitation.
  arXiv  Detail & Related papers  (2024-04-19T09:10:29Z)
• Character is Destiny: Can Role-Playing Language Agents Make Persona-Driven Decisions? [59.0123596591807]
  We benchmark the ability of Large Language Models (LLMs) in persona-driven decision-making. We investigate whether LLMs can predict characters' decisions provided by the preceding stories in high-quality novels. The results demonstrate that state-of-the-art LLMs exhibit promising capabilities in this task, yet substantial room for improvement remains.
  arXiv  Detail & Related papers  (2024-04-18T12:40:59Z)
• Identifying and Analyzing Performance-Critical Tokens in Large Language Models [52.404072802235234]
  We study how large language models learn to perform tasks from demonstrations. Our work sheds light on how large language models learn to perform tasks from demonstrations and deepens our understanding of the roles different types of tokens play in large language models.
  arXiv  Detail & Related papers  (2024-01-20T20:55:21Z)
• Large Language Models Meet Harry Potter: A Bilingual Dataset for Aligning Dialogue Agents with Characters [70.84938803753062]
  We introduce the Harry Potter Dialogue dataset, designed to advance the study of dialogue agents and character alignment. The dataset encompasses all dialogue sessions (in both English and Chinese) from the Harry Potter series. It is annotated with vital background information, including dialogue scenes, speakers, character relationships, and attributes.
  arXiv  Detail & Related papers  (2022-11-13T10:16:39Z)
• "Let Your Characters Tell Their Story": A Dataset for Character-Centric Narrative Understanding [31.803481510886378]
  We present LiSCU -- a new dataset of literary pieces and their summaries paired with descriptions of characters that appear in them. We also introduce two new tasks on LiSCU: Character Identification and Character Description Generation. Our experiments with several pre-trained language models adapted for these tasks demonstrate that there is a need for better models of narrative comprehension.
  arXiv  Detail & Related papers  (2021-09-12T06:12:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.