Do Large Language Models Perform Well on Comprehending Poetic Logic in Modern Chinese Poetry?

• URL: http://arxiv.org/abs/2608.21827v1
• Date: Sat, 22 Aug 2026 07:52:00 GMT
• Title: Do Large Language Models Perform Well on Comprehending Poetic Logic in Modern Chinese Poetry?
• Abstract summary: Large Language Models (LLMs) have achieved significant progress across a wide range of natural language processing (NLP) tasks.<n>Modern Chinese poetry necessitate a distinct form of reasoning for effective comprehension.<n>We propose Peony, the first benchmark specifically designed for evaluating the poetic logic of modern Chinese poetry.
• Score: 47.71394430680399
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have achieved significant progress across a wide range of natural language processing (NLP) tasks, yet their ability to understand literary texts, particularly modern Chinese poetry, remains largely unexplored. The unique literary characteristics of modern Chinese poetry necessitate a distinct form of reasoning for effective comprehension. Unlike conventional texts that convey clear information, the unique "poetic logic" of modern Chinese poetry requires a holistic reasoning approach that goes beyond superficial semantic analysis to be understood. However, current evaluation paradigms largely ignore this critical dimension. To address this gap, we propose Peony, the first benchmark specifically designed for evaluating the poetic logic of modern Chinese poetry. We define poetic logic as four tasks across three levels, namely stanza, line, and imagery, and systematically evaluate and analyze six mainstream LLMs based on Peony. We evaluate these models under both non-thinking and thinking configurations. The experimental results reveal the limitations of current LLMs in understanding the poetic logic of modern Chinese poetry and validate the effectiveness and necessity of Peony. Our data and code will be available.

Related papers

• Jiuge-Tuiqiao: An Interpretable Human-AI System for Classical Chinese Poetry Refinement [48.293435958731614]
  Jiuge-Tuiqiao is an interactive human-AI collaborative system for classical Chinese poetry composition.<n>Users can lock characters or lines, receive real-time prosody feedback, and obtain interpretable refinement suggestions.<n>Preliminary experiments and user feedback suggest that Jiuge-Tuiqiao improves controllability, interpretability, and user engagement.
  arXiv  Detail & Related papers  (2026-08-24T11:05:09Z)
• Who Wrote This Line? Evaluating the Detection of LLM-Generated Classical Chinese Poetry [45.27531384029669]
  AI-generated literary creations have raised prominent issues of creative authenticity and ethics in literary world.<n>Previous works have made significant progress in detecting AI-generated text, but it has yet to address classical Chinese poetry.<n>We introduce ChangAn, a benchmark for detecting LLM-generated classical Chinese poetry.
  arXiv  Detail & Related papers  (2026-04-11T08:52:08Z)
• Liaozhai through the Looking-Glass: On Paratextual Explicitation of Culture-Bound Terms in Machine Translation [70.43884512651668]
  We formalize Genette's (1987) theory of paratexts from literary and translation studies to introduce the task of paratextual explicitation for machine translation.<n>We construct a dataset of 560 expert-aligned paratexts from four English translations of the classical Chinese short story collection Liaozhai.<n>Our findings demonstrate the potential of paratextual explicitation in advancing machine translation beyond linguistic equivalence.
  arXiv  Detail & Related papers  (2025-09-27T16:27:36Z)
• Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry [37.86155340473244]
  It is difficult to identify whether a poem originated from humans or AI.<n>This paper proposes a novel benchmark for detecting AI-generated modern Chinese poetry.
  arXiv  Detail & Related papers  (2025-09-01T17:01:45Z)
• Picturized and Recited with Dialects: A Multimodal Chinese Representation Framework for Sentiment Analysis of Classical Chinese Poetry [7.374104697960381]
  We propose a dialect-enhanced multimodal framework for classical Chinese poetry sentiment analysis.<n>We extract sentence-level audio features from the poetry and incorporate audio from multiple dialects.<n>Our framework outperforms state-of-the-art methods on two public datasets.
  arXiv  Detail & Related papers  (2025-05-19T14:58:44Z)
• Understanding Literary Texts by LLMs: A Case Study of Ancient Chinese Poetry [9.970908656435066]
  In genres such as poetry, jokes, and short stories, numerous AI tools have emerged, offering refreshing new perspectives. evaluating literary works is often complex and hard to fully quantify, which directly hinders the further development of AI creation. This paper attempts to explore the mysteries of literary texts from the perspective of large language models.
  arXiv  Detail & Related papers  (2024-08-22T04:25:06Z)
• CCPM: A Chinese Classical Poetry Matching Dataset [50.90794811956129]
  We propose a novel task to assess a model's semantic understanding of poetry by poem matching. This task requires the model to select one line of Chinese classical poetry among four candidates according to the modern Chinese translation of a line of poetry. To construct this dataset, we first obtain a set of parallel data of Chinese classical poetry and modern Chinese translation.
  arXiv  Detail & Related papers  (2021-06-03T16:49:03Z)
• Generating Major Types of Chinese Classical Poetry in a Uniformed Framework [88.57587722069239]
  We propose a GPT-2 based framework for generating major types of Chinese classical poems. Preliminary results show this enhanced model can generate Chinese classical poems of major types with high quality in both form and content.
  arXiv  Detail & Related papers  (2020-03-13T14:16:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.