SpreadsheetArena: Decomposing Preference in LLM Generation of Spreadsheet Workbooks

• URL: http://arxiv.org/abs/2603.10002v1
• Date: Mon, 16 Feb 2026 14:24:36 GMT
• Title: SpreadsheetArena: Decomposing Preference in LLM Generation of Spreadsheet Workbooks
• Abstract summary: Large language models (LLMs) are increasingly tasked with producing and manipulating structured artifacts.<n>We introduce SpreadsheetArena, a platform for evaluating models' performance on the task.<n>We observe that stylistic, structural, and functional features of preferred spreadsheets vary substantially across use cases.
• Score: 8.306042091008957
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large language models (LLMs) are increasingly tasked with producing and manipulating structured artifacts. We consider the task of end-to-end spreadsheet generation, where language models are prompted to produce spreadsheet artifacts to satisfy users' explicit and implicit constraints, specified in natural language. We introduce SpreadsheetArena, a platform for evaluating models' performance on the task via blind pairwise evaluations of LLM-generated spreadsheet workbooks. As with other complex, open-ended tasks, relevant evaluation criteria can vary substantially across use cases and prompts, often in ways that are difficult to formalize. Compared to general chat or text generation settings, spreadsheet generation presents unique challenges and opportunities: the task output structure is well-defined and multi-dimensional, and there are often complex considerations around interactivity and layout. Among other findings, we observe that stylistic, structural, and functional features of preferred spreadsheets vary substantially across use cases, and expert evaluations of spreadsheets for finance prompts suggests that even highly ranked arena models do not reliably produce spreadsheets aligned with domain-specific best practices. Our hope is that our work prompts further study of end-to-end spreadsheet generation as a challenging and interesting category of complex, open-ended tasks for LLMs. Our live arena is hosted at https://spreadsheetarena.ai.

Related papers

• SheetCompass: Hierarchical Relation Graphs for Agentic Spreadsheet Reasoning [13.891450098227573]
  We propose Sheet, a graph-guided and memory-driven framework for spreadsheet reasoning and automation.<n>Sheet explicitly models structural relationships within and across worksheets while maintaining task-relevant information in memory.
  arXiv  Detail & Related papers  (2026-08-14T16:39:16Z)
• SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows [14.11925465537949]
  We introduce textscSpreadsheetBench 2, a taxonomy-level benchmark for spreadsheet agents.<n>The benchmark is constructed from authentic business data, including financial reports and corporate filings.<n>Results show that current systems remain far from reliable on real-world use.
  arXiv  Detail & Related papers  (2026-06-29T08:33:52Z)
• The Power of Order: Fooling LLMs with Adversarial Table Permutations [49.84535222399142]
  Adversarial Table Permutation is a gradient-based attack that efficiently identifies worst-case permutations designed to maximally disrupt model performance.<n>Our experiments demonstrate that ATP significantly degrades the performance of a wide range of LLMs.<n>Our findings expose a fundamental weakness in how current LLMs process structured data, underscoring the urgent need to develop permutation-robust models for reliable, real-world applications.
  arXiv  Detail & Related papers  (2026-05-01T06:25:55Z)
• SODBench: A Large Language Model Approach to Documenting Spreadsheet Operations [1.3669571918482655]
  This paper introduces Spreadsheet Operations Documentation (SOD), an AI task that involves generating human-readable explanations from spreadsheet operations.<n>We present a benchmark of 111 spreadsheet manipulation code snippets, each paired with a corresponding natural language summary.<n>Our findings suggest that LLMs can generate accurate spreadsheet documentation, making SOD a feasible prerequisite step toward enhancing, maintainability, collaborative in spreadsheets.
  arXiv  Detail & Related papers  (2025-10-22T01:36:13Z)
• Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction [80.88654868264645]
  Arranged and Organized Extraction Benchmark designed to evaluate ability of large language models to comprehend fragmented documents.<n>AOE includes 11 carefully crafted tasks across three diverse domains, requiring models to generate context-specific schema tailored to varied input queries.<n>Results show that even the most advanced models struggled significantly.
  arXiv  Detail & Related papers  (2025-07-22T06:37:51Z)
• Large Language Models for Spreadsheets: Benchmarking Progress and Evaluating Performance with FLARE [0.0]
  Large Language Models (LLMs) have demonstrated some significant capabilities across various domains.<n>This study introduces a benchmark framework to evaluate the performance of leading LLMs in executing spreadsheet functions.
  arXiv  Detail & Related papers  (2025-06-19T03:47:38Z)
• SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning [70.21358720599821]
  Large language models (LLMs) hold the promise of solving diverse tasks when provided with appropriate natural language prompts. We propose SELF-GUIDE, a multi-stage mechanism in which we synthesize task-specific input-output pairs from the student LLM. We report an absolute improvement of approximately 15% for classification tasks and 18% for generation tasks in the benchmark's metrics.
  arXiv  Detail & Related papers  (2024-07-16T04:41:58Z)
• SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation [34.8332394229927]
  SpreadsheetBench is designed to immerse current large language models (LLMs) in the actual workflow of spreadsheet users. Unlike existing benchmarks that rely on synthesized queries and simplified spreadsheet files, SpreadsheetBench is built from 912 real questions gathered from online Excel forums. Our comprehensive evaluation of various LLMs under both single-round and multi-round inference settings reveals a substantial gap between the state-of-the-art (SOTA) models and human performance.
  arXiv  Detail & Related papers  (2024-06-21T09:06:45Z)
• SheetAgent: Towards A Generalist Agent for Spreadsheet Reasoning and Manipulation via Large Language Models [45.930510174309845]
  Large language model (LLM) has been recently attempted for automatic spreadsheet manipulation.<n>SheetAgent consists of three collaborative modules: Planner, Informer, and Retriever.<n>Extensive experiments demonstrate that SheetAgent delivers 20--40% pass rate improvements on multiple benchmarks over baselines.
  arXiv  Detail & Related papers  (2024-03-06T11:48:08Z)
• TAP4LLM: Table Provider on Sampling, Augmenting, and Packing Semi-structured Data for Large Language Model Reasoning [55.33939289989238]
  We propose TAP4LLM as a versatile pre-processor suite for leveraging large language models (LLMs) in table-based tasks effectively. It covers several distinct components: (1) table sampling to decompose large tables into manageable sub-tables based on query semantics, (2) table augmentation to enhance tables with additional knowledge from external sources or models, and (3) table packing & serialization to convert tables into various formats suitable for LLMs' understanding.
  arXiv  Detail & Related papers  (2023-12-14T15:37:04Z)
• SpreadsheetCoder: Formula Prediction from Semi-structured Context [70.41579328458116]
  We propose a BERT-based model architecture to represent the tabular context in both row-based and column-based formats. We train our model on a large dataset of spreadsheets, and demonstrate that SpreadsheetCoder achieves top-1 prediction accuracy of 42.51%. Compared to the rule-based system, SpreadsheetCoder 82% assists more users in composing formulas on Google Sheets.
  arXiv  Detail & Related papers  (2021-06-26T11:26:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.