METRICALARGS: A Taxonomy for Studying Metrical Poetry with LLMs

• URL: http://arxiv.org/abs/2510.08188v1
• Date: Thu, 09 Oct 2025 13:14:38 GMT
• Title: METRICALARGS: A Taxonomy for Studying Metrical Poetry with LLMs
• Authors: Chalamalasetti Kranti, Sowmya Vajjala,
• Abstract summary: We introduce MetricalARGS, the first taxonomy of poetry-related NLP tasks designed to evaluate Large Language Models.<n>We discuss how these tasks relate to existing NLP tasks, addressing questions around datasets and evaluation metrics.
• Score: 4.33144664431421
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Prior NLP work studying poetry has focused primarily on automatic poem generation and summarization. Many languages have well-studied traditions of poetic meter which enforce constraints on a poem in terms of syllable and phoneme patterns. Such advanced literary forms offer opportunities for probing deeper reasoning and language understanding in Large Language Models (LLMs) and their ability to follow strict pre-requisites and rules. In this paper, we introduce MetricalARGS, the first taxonomy of poetry-related NLP tasks designed to evaluate LLMs on metrical poetry across four dimensions: Analysis, Retrieval, Generation, and Support. We discuss how these tasks relate to existing NLP tasks, addressing questions around datasets and evaluation metrics. Taking Telugu as our example language, we illustrate how the taxonomy can be used in practice. MetricalARGS highlights the broader possibilities for understanding the capabilities and limitations of today's LLMs through the lens of metrical poetry.

Related papers

• Tell, Don't Show: Leveraging Language Models' Abstractive Retellings to Model Literary Themes [9.471374217162843]
  We propose Retell, a simple, accessible topic modeling approach for literature.<n>We prompt resource-efficient, generative language models (LMs) to tell what passages show.
  arXiv  Detail & Related papers  (2025-05-29T06:59:21Z)
• A Bayesian Approach to Harnessing the Power of LLMs in Authorship Attribution [57.309390098903]
  Authorship attribution aims to identify the origin or author of a document. Large Language Models (LLMs) with their deep reasoning capabilities and ability to maintain long-range textual associations offer a promising alternative. Our results on the IMDb and blog datasets show an impressive 85% accuracy in one-shot authorship classification across ten authors.
  arXiv  Detail & Related papers  (2024-10-29T04:14:23Z)
• Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning Tasks [68.33068005789116]
  We introduce ReDial, a benchmark containing 1.2K+ parallel query pairs in Standardized English and AAVE.<n>We evaluate widely used models, including GPT, Claude, Llama, Mistral, and the Phi model families.<n>Our work establishes a systematic and objective framework for analyzing LLM bias in dialectal queries.
  arXiv  Detail & Related papers  (2024-10-14T18:44:23Z)
• Understanding Literary Texts by LLMs: A Case Study of Ancient Chinese Poetry [9.970908656435066]
  In genres such as poetry, jokes, and short stories, numerous AI tools have emerged, offering refreshing new perspectives. evaluating literary works is often complex and hard to fully quantify, which directly hinders the further development of AI creation. This paper attempts to explore the mysteries of literary texts from the perspective of large language models.
  arXiv  Detail & Related papers  (2024-08-22T04:25:06Z)
• Sonnet or Not, Bot? Poetry Evaluation for Large Models and Datasets [3.0040661953201475]
  Large language models (LLMs) can now generate and recognize poetry. We develop a task to evaluate how well LLMs recognize one aspect of English-language poetry. We show that state-of-the-art LLMs can successfully identify both common and uncommon fixed poetic forms.
  arXiv  Detail & Related papers  (2024-06-27T05:36:53Z)
• Categorical Syllogisms Revisited: A Review of the Logical Reasoning Abilities of LLMs for Analyzing Categorical Syllogism [62.571419297164645]
  This paper provides a systematic overview of prior works on the logical reasoning ability of large language models for analyzing categorical syllogisms.<n>We first investigate all the possible variations for the categorical syllogisms from a purely logical perspective.<n>We then examine the underlying configurations (i.e., mood and figure) tested by the existing datasets.
  arXiv  Detail & Related papers  (2024-06-26T21:17:20Z)
• LFED: A Literary Fiction Evaluation Dataset for Large Language Models [58.85989777743013]
  We collect 95 literary fictions that are either originally written in Chinese or translated into Chinese, covering a wide range of topics across several centuries. We define a question taxonomy with 8 question categories to guide the creation of 1,304 questions. We conduct an in-depth analysis to ascertain how specific attributes of literary fictions (e.g., novel types, character numbers, the year of publication) impact LLM performance in evaluations.
  arXiv  Detail & Related papers  (2024-05-16T15:02:24Z)
• PhonologyBench: Evaluating Phonological Skills of Large Language Models [57.80997670335227]
  Phonology, the study of speech's structure and pronunciation rules, is a critical yet often overlooked component in Large Language Model (LLM) research. We present PhonologyBench, a novel benchmark consisting of three diagnostic tasks designed to explicitly test the phonological skills of LLMs. We observe a significant gap of 17% and 45% on Rhyme Word Generation and Syllable counting, respectively, when compared to humans.
  arXiv  Detail & Related papers  (2024-04-03T04:53:14Z)
• A Computational Approach to Style in American Poetry [19.41186389974801]
  We develop a method to assess the style of American poems and to visualize a collection of poems in relation to one another. qualitative poetry criticism helped guide our development of metrics that analyze various orthographic, syntactic, and phonemic features. Our method has potential applications to academic research of texts, to research of the intuitive personal response to poetry, and to making recommendations to readers based on their favorite poems.
  arXiv  Detail & Related papers  (2023-10-13T18:49:14Z)
• Natural Language Decompositions of Implicit Content Enable Better Text Representations [52.992875653864076]
  We introduce a method for the analysis of text that takes implicitly communicated content explicitly into account.<n>We use a large language model to produce sets of propositions that are inferentially related to the text that has been observed.<n>Our results suggest that modeling the meanings behind observed language, rather than the literal text alone, is a valuable direction for NLP.
  arXiv  Detail & Related papers  (2023-05-23T23:45:20Z)
• CCPM: A Chinese Classical Poetry Matching Dataset [50.90794811956129]
  We propose a novel task to assess a model's semantic understanding of poetry by poem matching. This task requires the model to select one line of Chinese classical poetry among four candidates according to the modern Chinese translation of a line of poetry. To construct this dataset, we first obtain a set of parallel data of Chinese classical poetry and modern Chinese translation.
  arXiv  Detail & Related papers  (2021-06-03T16:49:03Z)
• Metrical Tagging in the Wild: Building and Annotating Poetry Corpora with Rhythmic Features [0.0]
  We provide large poetry corpora for English and German, and annotate prosodic features in smaller corpora to train corpus driven neural models. We show that BiLSTM-CRF models with syllable embeddings outperform a CRF baseline and different BERT-based approaches.
  arXiv  Detail & Related papers  (2021-02-17T16:38:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.