CraftAlign: Feature-Grounded Evaluation and Revision Guidance for AI Stories

• URL: http://arxiv.org/abs/2608.01377v1
• Date: Sun, 02 Aug 2026 16:57:05 GMT
• Title: CraftAlign: Feature-Grounded Evaluation and Revision Guidance for AI Stories
• Abstract summary: We introduce CraftAlign, a framework that aligns AI stories with the craft of human storytelling.<n>A feature estimator built on Qwen3.5-9B predicts 304 explicit writing features spanning style and narrative.<n>At inference time, CraftAlign applies schema-valid structured perturbations, selects changes that move the feature configuration toward the Human writing pattern.
• Score: 7.837987786613499
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models can now generate fluent and complete stories, yet many outputs still feel formulaic and unnatural because of cliches, over-explanation, linear causal progression, and stereotyped endings, an immediately recognizable AI flavor. Existing detection and evaluation methods often stop at source labels or holistic scores, while revision methods typically target predefined issues through localized edits, limiting their ability to support multiple plausible revision strategies or guide story-wide changes in information release, causal organization, and ending treatment. We introduce CraftAlign, a framework that aligns AI stories with the craft of human storytelling by both assessing Human/AI writing patterns and providing revision guidance. CraftAlign comprises two learned modules and an inference-time guidance pipeline. A feature estimator built on Qwen3.5-9B predicts 304 explicit writing features spanning style and narrative. A class-conditional energy model scores the resulting feature configuration against Human and AI writing patterns, conditioning on the original writing prompt when available. At inference time, CraftAlign applies schema-valid structured perturbations, selects changes that move the feature configuration toward the Human writing pattern, and converts them into natural-language guidance for a separate editor to rewrite the full story. Experiments show that CraftAlign accurately distinguishes Human and AI writing patterns and that its guidance outperforms revision baselines across editors and in a human study.

Related papers

• SciDiagramEdit: Learning to Edit Scientific Diagrams from Paper Revisions [66.46898035008782]
  We present SciDiagramEdit, a benchmark and skill-evolution framework that learns from natural paper revisions.<n>An agentic proposer continually refines the agent's skill specification from execution traces over multiple epochs.<n>The resulting skill progressively lifts edit accuracy on a held-out validation set.
  arXiv  Detail & Related papers  (2026-07-16T17:58:36Z)
• Operation-Guided Progressive Human-to-AI Text Transformation Benchmark for Multi-Granularity AI-Text Detection [49.41415417307568]
  OpAI-Bench is an operation-guided benchmark for studying progressive human-to-AI text transformation.<n>We show that AI-text detectability is governed not only by the proportion of AI-edited content, but also by edit operation, domain, and cumulative revision history.
  arXiv  Detail & Related papers  (2026-06-04T17:58:05Z)
• Explainable Disentangled Representation Learning for Generalizable Authorship Attribution in the Era of Generative AI [70.05655131235999]
  Explainable Authorship Variational Autoencoder (EAVAE) is a novel framework that explicitly disentangles style from content.<n>Disentanglement is enforced through a novel discriminator.<n>EAVAE excels in few-shot learning over the M4 dataset for AI-generated text detection.
  arXiv  Detail & Related papers  (2026-04-23T05:34:20Z)
• Generation, Evaluation, and Explanation of Novelists' Styles with Single-Token Prompts [3.7189423451031356]
  We present a framework for both generating and evaluating sentences in the style of 19th-century novelists.<n>Large language models are fine-tuned with minimal, single-token prompts to produce text in the voices of authors such as Dickens, Austen, Twain, Alcott, and Melville.
  arXiv  Detail & Related papers  (2025-11-25T16:25:44Z)
• Learning to Reason for Long-Form Story Generation [84.09733333295338]
  We propose a general story-generation task (Next-Chapter Prediction) and a reward formulation (Verified Rewards via Completion Likelihood Improvement)<n>We learn to reason over a story's condensed information and generate a detailed plan for the next chapter.<n>Our reasoning is evaluated via the chapters it helps a story-generator create, and compared against non-trained and supervised finetuning (SFT) baselines.
  arXiv  Detail & Related papers  (2025-03-28T18:48:26Z)
• To Revise or Not to Revise: Learning to Detect Improvable Claims for Argumentative Writing Support [20.905660642919052]
  We explore the main challenges to identifying argumentative claims in need of specific revisions. We propose a new sampling strategy based on revision distance. We provide evidence that using contextual information and domain knowledge can further improve prediction results.
  arXiv  Detail & Related papers  (2023-05-26T10:19:54Z)
• EditEval: An Instruction-Based Benchmark for Text Improvements [73.5918084416016]
  This work presents EditEval: An instruction-based, benchmark and evaluation suite for automatic evaluation of editing capabilities. We evaluate several pre-trained models, which shows that InstructGPT and PEER perform the best, but that most baselines fall below the supervised SOTA. Our analysis shows that commonly used metrics for editing tasks do not always correlate well, and that optimization for prompts with the highest performance does not necessarily entail the strongest robustness to different models.
  arXiv  Detail & Related papers  (2022-09-27T12:26:05Z)
• PEER: A Collaborative Language Model [70.11876901409906]
  We introduce PEER, a collaborative language model that imitates the entire writing process itself. PEER can write drafts, add suggestions, propose edits and provide explanations for its actions. We show that PEER achieves strong performance across various domains and editing tasks.
  arXiv  Detail & Related papers  (2022-08-24T16:56:47Z)
• Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revision [11.495407637511878]
  We present a human-in-the-loop iterative text revision system, Read, Revise, Repeat (R3) R3 aims at achieving high quality text revisions with minimal human efforts by reading model-generated revisions and user feedbacks, revising documents, and repeating human-machine interactions.
  arXiv  Detail & Related papers  (2022-04-07T18:33:10Z)
• Understanding Iterative Revision from Human-Written Text [10.714872525208385]
  IteraTeR is the first large-scale, multi-domain, edit-intention annotated corpus of iteratively revised text. We better understand the text revision process, making vital connections between edit intentions and writing quality.
  arXiv  Detail & Related papers  (2022-03-08T01:47:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.