SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows

• URL: http://arxiv.org/abs/2606.29955v1
• Date: Mon, 29 Jun 2026 08:33:52 GMT
• Title: SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflows
• Abstract summary: We introduce textscSpreadsheetBench 2, a taxonomy-level benchmark for spreadsheet agents.<n>The benchmark is constructed from authentic business data, including financial reports and corporate filings.<n>Results show that current systems remain far from reliable on real-world use.
• Score: 14.11925465537949
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Spreadsheets are widely used for business analysis, financial modeling, reporting, and decision-making. However, most existing spreadsheet benchmarks evaluate isolated operations such as single-formula generation or local cell edits, and therefore fail to capture end-to-end workflows in realistic business settings. We introduce \textsc{SpreadsheetBench 2}, a workflow-level benchmark for spreadsheet agents that covers three task categories: generation, debugging, and visualization. The benchmark is constructed from authentic business data, including financial reports and corporate filings, and is annotated and validated by domain experts. The benchmark contains 321 tasks; each instance averages 11.8 worksheets and requires 593.5 cell modifications, reflecting large multi-sheet workbooks with cross-sheet dependencies. We evaluate eight frontier large language models under a unified multi-turn agent scaffold, and additionally include several LLM-based spreadsheet products as complementary baselines. Results show that current systems remain far from reliable on real-world workflows: the best model achieves 34.89\% overall task accuracy, and debugging accuracy is as low as 12.00\%. Trajectory analysis and a failure taxonomy further indicate that insufficient spreadsheet inspection and incorrect target-cell selection are the dominant bottlenecks. Together, these findings position \textsc{SpreadsheetBench 2} as a challenging testbed for advancing reliable spreadsheet automation. Project page: https://spreadsheetbench.github.io/

Related papers

• Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction [57.138342889101345]
  Tencent WorkBuddy Bench is a multi-domain evaluation suite for coding agents.<n>This report documents its construction methodology, scoring protocol, and a cross-model leaderboard.
  arXiv  Detail & Related papers  (2026-07-23T04:34:06Z)
• WorkstreamBench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance [4.787072076364137]
  frontier AI labs have developed agents that can construct entire spreadsheets from scratch.<n>This is especially relevant in finance, where core such as financial modeling, forecasting, and scenario analysis are commonly conducted through spreadsheets.<n>Existing spreadsheet benchmarks do not measure this advanced capability, focusing instead on question-answering or single-formula edits.
  arXiv  Detail & Related papers  (2026-05-21T16:06:34Z)
• Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Reasoning [43.91509663025854]
  Spreadsheets are central to real-world applications such as enterprise reporting, auditing, and scientific data management.<n>Existing large language model based approaches typically treat tables as plain text, overlooking critical layout cues and visual semantics.<n>We propose a two-stage multi-agent framework for spreadsheet understanding that adopts a step-by-step reading and reasoning paradigm.
  arXiv  Detail & Related papers  (2026-04-14T04:47:21Z)
• Beyond Rows to Reasoning: Agentic Retrieval for Multimodal Spreadsheet Understanding and Editing [0.7723674433972977]
  We introduce Beyond Rows to Reasoning (BRTR), a framework for spreadsheet understanding that replaces single-pass retrieval with an iterative tool-calling loop.<n>BRTR achieves state-of-the-art performance across three frontier spreadsheet understanding benchmarks.
  arXiv  Detail & Related papers  (2026-03-06T17:36:39Z)
• SpreadsheetArena: Decomposing Preference in LLM Generation of Spreadsheet Workbooks [8.306042091008957]
  Large language models (LLMs) are increasingly tasked with producing and manipulating structured artifacts.<n>We introduce SpreadsheetArena, a platform for evaluating models' performance on the task.<n>We observe that stylistic, structural, and functional features of preferred spreadsheets vary substantially across use cases.
  arXiv  Detail & Related papers  (2026-02-16T14:24:36Z)
• SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation [34.8332394229927]
  SpreadsheetBench is designed to immerse current large language models (LLMs) in the actual workflow of spreadsheet users. Unlike existing benchmarks that rely on synthesized queries and simplified spreadsheet files, SpreadsheetBench is built from 912 real questions gathered from online Excel forums. Our comprehensive evaluation of various LLMs under both single-round and multi-round inference settings reveals a substantial gap between the state-of-the-art (SOTA) models and human performance.
  arXiv  Detail & Related papers  (2024-06-21T09:06:45Z)
• TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios [51.66718740300016]
  TableLLM is a robust large language model (LLM) with 8 billion parameters.<n>TableLLM is purpose-built for proficiently handling data manipulation tasks.<n>We have released the model checkpoint, source code, benchmarks, and a web application for user interaction.
  arXiv  Detail & Related papers  (2024-03-28T11:21:12Z)
• SheetAgent: Towards A Generalist Agent for Spreadsheet Reasoning and Manipulation via Large Language Models [45.930510174309845]
  Large language model (LLM) has been recently attempted for automatic spreadsheet manipulation.<n>SheetAgent consists of three collaborative modules: Planner, Informer, and Retriever.<n>Extensive experiments demonstrate that SheetAgent delivers 20--40% pass rate improvements on multiple benchmarks over baselines.
  arXiv  Detail & Related papers  (2024-03-06T11:48:08Z)
• Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers [121.53749383203792]
  We present a holistic end-to-end solution for annotating the factuality of large language models (LLMs)-generated responses. We construct an open-domain document-level factuality benchmark in three-level granularity: claim, sentence and document. Preliminary experiments show that FacTool, FactScore and Perplexity are struggling to identify false claims.
  arXiv  Detail & Related papers  (2023-11-15T14:41:57Z)
• SpreadsheetCoder: Formula Prediction from Semi-structured Context [70.41579328458116]
  We propose a BERT-based model architecture to represent the tabular context in both row-based and column-based formats. We train our model on a large dataset of spreadsheets, and demonstrate that SpreadsheetCoder achieves top-1 prediction accuracy of 42.51%. Compared to the rule-based system, SpreadsheetCoder 82% assists more users in composing formulas on Google Sheets.
  arXiv  Detail & Related papers  (2021-06-26T11:26:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.