Detecting and Guiding LLM-Generated Korean Poetry with Interpretable Form-level Features

• URL: http://arxiv.org/abs/2608.28986v1
• Date: Sat, 29 Aug 2026 01:45:50 GMT
• Title: Detecting and Guiding LLM-Generated Korean Poetry with Interpretable Form-level Features
• Abstract summary: We quantify the human-LLM gap along four form-level linguistic dimensions.<n>We operationalize these dimensions as five interpretable features.<n>For generation, expert evaluation on GPT-5.2 prefers feature-guided poems over the unconstrained baseline.
• Score: 17.99472063920348
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLMs often struggle with modern Korean poetry, producing outputs that resemble "line-broken prose." We address two coupled tasks: detecting whether a Korean poem is human- or LLM-authored, and guiding LLMs to generate poetry closer in form to human writing. We quantify the human-LLM gap along four form-level linguistic dimensions: output length (Volume), the diversity and connective use of line-final forms (Structure Variation), the irregularity of line lengths (Rhythmic Irregularity), and adherence to standard orthography (Normative Adherence). We operationalize these dimensions as five interpretable features. For detection, a logistic regression classifier over these five features attains an average AUC-ROC of 83.60 in zero-shot out-of-distribution detection across seven unseen LLMs, versus 75.84 for the strongest baseline in our comparison, KatFishNet, an absolute gain of 7.76 AUC points and a 10.23% relative improvement; one generator-specific punctuation pattern outside our taxonomy remains a boundary case. For generation, expert evaluation on GPT-5.2 prefers feature-guided poems over the unconstrained baseline, and analyses across GPT-5.2 and Gemini-3 show that targeted length, rhythm, and ending statistics move toward the human distribution. These results suggest that interpretable, language-specific features can bridge the diagnosis and guidance of LLM-generated poetry.

Related papers

• System Report for CCL25-Eval Task 5: New Dataset and LoRA-Fine-Tuned Qwen2.5 [0.0]
  We decompose the task into three subtasks: term interpretation, semantic interpretation, and emotional inference.<n>Based on multiple open-source datasets, we perform data cleansing and alignment to construct the Classical Chinese Poetry Instruction Pair dataset.<n>We then propose a domain-specialized LLM, called PoetryQwen, by applying Low-Rank Adaptation (LoRA) to fine-tune the Qwen2.5-14B model.
  arXiv  Detail & Related papers  (2026-06-10T17:54:32Z)
• From Context Shift to Stylistic Collapse: Why Training Objectives Matter More Than Scale [1.2328006781720982]
  Language models trained with contemporary pipelines exhibit severe reshaping of linguistic features, leading to extreme language re-distribution.<n>We analyze 17 models (410M-100B+ parameters) across 24 linguistically-motivated probes.<n>Our work reveals a structural limitation of current alignment pipelines, with implications for AI detection, training data contamination, and long-term linguistic evolution.
  arXiv  Detail & Related papers  (2026-04-08T02:13:46Z)
• From Intuition to Expertise: Rubric-Based Cognitive Calibration for Human Detection of LLM-Generated Korean Text [7.067247930901556]
  We introduce LREAD, a rubric derived from national Korean writing standards.<n>Phase 1 measures intuition-only detection, Phase 2 enforces criterion-level scoring, and Phase 3 evaluates domain-focused mastery.<n>Across phases, majority-vote accuracy increases from 60% to 100%, accompanied by stronger inter-annotator agreement.
  arXiv  Detail & Related papers  (2026-01-06T10:51:39Z)
• Protein as a Second Language for LLMs [50.34983283157322]
  "Protein-as-Second-Language" framework reformulates amino-acid sequences as sentences in a novel symbolic language.<n>We curate a bilingual corpus of 79,926 protein-QA instances spanning attribute prediction, descriptive understanding, and extended reasoning.<n>Our method delivers consistent gains across diverse open-source LLMs and GPT-4, achieving up to 17.2% ROUGE-L improvement.
  arXiv  Detail & Related papers  (2025-10-13T09:21:45Z)
• How Well Do LLMs Imitate Human Writing Style? [2.3754840025365183]
  Large language models (LLMs) can generate fluent text, but their ability to replicate the distinctive style of a specific human author remains unclear.<n>We present a fast, training-free framework for authorship verification and style imitation analysis.<n>It achieves 97.5% accuracy on academic essays and 94.5% in cross-domain evaluation.
  arXiv  Detail & Related papers  (2025-09-29T15:34:40Z)
• When Punctuation Matters: A Large-Scale Comparison of Prompt Robustness Methods for LLMs [55.20230501807337]
  We present the first systematic evaluation of 5 methods for improving prompt robustness within a unified experimental framework.<n>We benchmark these techniques on 8 models from Llama, Qwen and Gemma families across 52 tasks from Natural Instructions dataset.
  arXiv  Detail & Related papers  (2025-08-15T10:32:50Z)
• Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-training [57.62126373849383]
  Cross-lingual In-context Pre-training (CrossIC-PT) is a simple and scalable approach that enhances cross-lingual transfer.<n>We construct CrossIC-PT samples by interleaving semantic-related bilingual Wikipedia documents into a single context window.<n> Experimental results demonstrate that CrossIC-PT improves multilingual performance on three models across six target languages.
  arXiv  Detail & Related papers  (2025-04-29T07:24:25Z)
• SLAM: Towards Efficient Multilingual Reasoning via Selective Language Alignment [78.4550589538805]
  We propose an efficient multilingual reasoning alignment approach that precisely identifies and fine-tunes the layers responsible for handling multilingualism.<n> Experimental results show that our method, SLAM, only tunes 6 layers' feed-forward sub-layers including 6.5-8% of all parameters within 7B and 13B LLMs.
  arXiv  Detail & Related papers  (2025-01-07T10:29:43Z)
• Hate Personified: Investigating the role of LLMs in content moderation [64.26243779985393]
  For subjective tasks such as hate detection, where people perceive hate differently, the Large Language Model's (LLM) ability to represent diverse groups is unclear. By including additional context in prompts, we analyze LLM's sensitivity to geographical priming, persona attributes, and numerical information to assess how well the needs of various groups are reflected.
  arXiv  Detail & Related papers  (2024-10-03T16:43:17Z)
• Pragmatic Competence Evaluation of Large Language Models for the Korean Language [0.6757476692230009]
  This study evaluates how well Large Language Models (LLMs) understand context-dependent expressions from a pragmatic standpoint, specifically in Korean. We use both Multiple-Choice Questions (MCQs) for automatic evaluation and Open-Ended Questions (OEQs) assessed by human experts.
  arXiv  Detail & Related papers  (2024-03-19T12:21:20Z)
• Whose LLM is it Anyway? Linguistic Comparison and LLM Attribution for GPT-3.5, GPT-4 and Bard [5.500249707065662]
  Large Language Models (LLMs) are capable of generating text that is similar to or surpasses human quality.<n>We compare the vocabulary, Part-Of-Speech (POS) distribution, dependency distribution, and sentiment of texts generated by three of the most popular LLMs to diverse inputs.<n>The results point to significant linguistic variations which, in turn, enable us to attribute a given text to its LLM origin with a favorable 88% accuracy.
  arXiv  Detail & Related papers  (2024-02-22T13:25:17Z)
• "Knowing When You Don't Know": A Multilingual Relevance Assessment Dataset for Robust Retrieval-Augmented Generation [90.09260023184932]
  Retrieval-Augmented Generation (RAG) grounds Large Language Model (LLM) output by leveraging external knowledge sources to reduce factual hallucinations. NoMIRACL is a human-annotated dataset for evaluating LLM robustness in RAG across 18 typologically diverse languages. We measure relevance assessment using: (i) hallucination rate, measuring model tendency to hallucinate, when the answer is not present in passages in the non-relevant subset, and (ii) error rate, measuring model inaccuracy to recognize relevant passages in the relevant subset.
  arXiv  Detail & Related papers  (2023-12-18T17:18:04Z)
• PoetryDiffusion: Towards Joint Semantic and Metrical Manipulation in Poetry Generation [58.36105306993046]
  Controllable text generation is a challenging and meaningful field in natural language generation (NLG) In this paper, we pioneer the use of the Diffusion model for generating sonnets and Chinese SongCi poetry. Our model outperforms existing models in automatic evaluation of semantic, metrical, and overall performance as well as human evaluation.
  arXiv  Detail & Related papers  (2023-06-14T11:57:31Z)
• INSTRUCTSCORE: Explainable Text Generation Evaluation with Finegrained Feedback [80.57617091714448]
  We present InstructScore, an explainable evaluation metric for text generation. We fine-tune a text evaluation metric based on LLaMA, producing a score for generated text and a human readable diagnostic report.
  arXiv  Detail & Related papers  (2023-05-23T17:27:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.