A Multi-Framework Comparison of Outline Stages in Long-Form Generation with LLMs

• URL: http://arxiv.org/abs/2608.26177v1
• Date: Thu, 13 Aug 2026 15:44:03 GMT
• Title: A Multi-Framework Comparison of Outline Stages in Long-Form Generation with LLMs
• Abstract summary: Long-form generation exposes fundamental limitations of large language models.<n>SuperWriter ranks first in the length-constrained single-chapter mode.<n> follow-up experiments will expand the sample size and add cross-model evaluators.
• Score: 9.826721433645526
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-form generation exposes fundamental limitations of large language models. Even 70B-parameter models exhibit length collapse at 16k-token outputs, and multi-chapter stories frequently trigger the attribute drift characteristic of the ``lost-in-the-middle'' effect. The ``outline-first, write-later'' paradigm has gained wide adoption, yet existing research evaluates the final writing rather than the outline itself, conflating two evaluation objects that should be decoupled. We construct a unified head-to-head benchmark covering 7 representative long-form generation frameworks across 3 generation granularities -- single-chapter, multi-chapter, and whole-book -- and propose an anchor-based LLM-as-a-judge protocol that directly assesses outlines against the source text on a 5-point anchored scale. Across 21 framework-granularity cells, no single framework dominates; performance depends on the match between a framework's intrinsic output form and the target granularity. SuperWriter ranks first in the length-constrained single-chapter mode, but this advantage degrades in whole-book mode. The outline-side ranking correlates only moderately with the writing-side ranking, supporting the outline--writing decoupling principle. Compute constraints limit the writing-side evaluation to a subset of cases; follow-up experiments will expand the sample size and add cross-model evaluators to enable stronger statistical inference.

Related papers

• Inferring the Size of Large Language Models From Popular Text Memorization [0.989901717499058]
  How accurately a model predicts the next word across text fragments of varying length is a reliable signal of how much it has memorized them.<n>We aggregate this memorization signal across a diverse corpus of texts and fragment lengths into a single accuracy profile vector per model.<n>We build two complementary inference methods on top of it: a pairwise statistical test that determines which of two models is larger, and a scaling-law estimator.
  arXiv  Detail & Related papers  (2026-05-28T01:20:35Z)
• HCRE: LLM-based Hierarchical Classification for Cross-Document Relation Extraction with a Prediction-then-Verification Strategy [54.91468501159335]
  Cross-document relation extraction (RE) aims to identify relations between the head tail entities located in different documents.<n>We propose an inference strategy that improves reliability through multi-view verification at each level.
  arXiv  Detail & Related papers  (2026-04-09T07:55:27Z)
• Toward General Semantic Chunking: A Discriminative Framework for Ultra-Long Documents [10.682600460180174]
  Long-document topic segmentation plays an important role in information retrieval and document understanding.<n>Traditional discriminative models are constrained by fixed windows and cannot model document-level semantics.<n>generative large language models can output paragraph boundaries, but inference is expensive and long inputs are difficult to support.
  arXiv  Detail & Related papers  (2025-12-23T16:16:51Z)
• WritingBench: A Comprehensive Benchmark for Generative Writing [87.48445972563631]
  We present WritingBench, a benchmark designed to evaluate large language models (LLMs) across 6 core writing domains and 100, encompassing creative, persuasive, informative, and technical writing.<n>We propose a query-dependent evaluation framework that empowers LLMs to dynamically generate instance-specific assessment criteria.<n>This framework is complemented by a fine-tuned critic model for criteria-aware scoring, enabling evaluations in style, format and length.
  arXiv  Detail & Related papers  (2025-03-07T08:56:20Z)
• Context-Aware Hierarchical Merging for Long Document Summarization [56.96619074316232]
  We propose different approaches to enrich hierarchical merging with context from the source document.<n> Experimental results on datasets representing legal and narrative domains show that contextual augmentation consistently outperforms zero-shot and hierarchical merging baselines.
  arXiv  Detail & Related papers  (2025-02-03T01:14:31Z)
• Long text outline generation: Chinese text outline based on unsupervised framework and large language mode [9.570650109953679]
  We propose a novel outline generation method for Chinese, combining an unsupervised framework with large models.<n>Specifically, the method first generates chapter feature graph data based on entity and syntactic dependency relationships.<n>A representation module based on graph attention layers learns deep embeddings of the chapter graph data to segment plot boundaries.<n>Finally, we employ a large model to generate summaries of each plot segment and produce the overall outline.
  arXiv  Detail & Related papers  (2024-12-01T13:46:15Z)
• GLIMMER: Incorporating Graph and Lexical Features in Unsupervised Multi-Document Summarization [13.61818620609812]
  We propose a lightweight yet effective unsupervised approach called GLIMMER: a Graph and LexIcal features based unsupervised Multi-docuMEnt summaRization approach. It first constructs a sentence graph from the source documents, then automatically identifies semantic clusters by mining low-level features from raw texts. Experiments conducted on Multi-News, Multi-XScience and DUC-2004 demonstrate that our approach outperforms existing unsupervised approaches.
  arXiv  Detail & Related papers  (2024-08-19T16:01:48Z)
• Long Document Summarization with Top-down and Bottom-up Inference [113.29319668246407]
  We propose a principled inference framework to improve summarization models on two aspects. Our framework assumes a hierarchical latent structure of a document where the top-level captures the long range dependency. We demonstrate the effectiveness of the proposed framework on a diverse set of summarization datasets.
  arXiv  Detail & Related papers  (2022-03-15T01:24:51Z)
• Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing [110.4684789199555]
  We introduce scenario-based semantic parsing: a variant of the original task which first requires disambiguating an utterance's "scenario" This formulation enables us to isolate coarse-grained and fine-grained aspects of the task, each of which we solve with off-the-shelf neural modules. Our model is modular, differentiable, interpretable, and allows us to garner extra supervision from scenarios.
  arXiv  Detail & Related papers  (2022-02-02T08:00:21Z)
• Unsupervised Summarization with Customized Granularities [76.26899748972423]
  We propose the first unsupervised multi-granularity summarization framework, GranuSum. By inputting different numbers of events, GranuSum is capable of producing multi-granular summaries in an unsupervised manner.
  arXiv  Detail & Related papers  (2022-01-29T05:56:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.