NL2Formula: Generating Spreadsheet Formulas from Natural Language Queries

• URL: http://arxiv.org/abs/2402.14853v1
• Date: Tue, 20 Feb 2024 05:58:05 GMT
• Title: NL2Formula: Generating Spreadsheet Formulas from Natural Language Queries
• Authors: Wei Zhao, Zhitao Hou, Siyuan Wu, Yan Gao, Haoyu Dong, Yao Wan, Hongyu Zhang, Yulei Sui, Haidong Zhang
• Abstract summary: This paper introduces a novel benchmark task called NL2Formula. The aim is to generate executable formulas that are grounded on a spreadsheet table, given a Natural Language (NL) query as input. We construct a comprehensive dataset consisting of 70,799 paired NL queries and corresponding spreadsheet formulas, covering 21,670 tables and 37 types of formula functions.
• Score: 29.33149993368329
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Writing formulas on spreadsheets, such as Microsoft Excel and Google Sheets, is a widespread practice among users performing data analysis. However, crafting formulas on spreadsheets remains a tedious and error-prone task for many end-users, particularly when dealing with complex operations. To alleviate the burden associated with writing spreadsheet formulas, this paper introduces a novel benchmark task called NL2Formula, with the aim to generate executable formulas that are grounded on a spreadsheet table, given a Natural Language (NL) query as input. To accomplish this, we construct a comprehensive dataset consisting of 70,799 paired NL queries and corresponding spreadsheet formulas, covering 21,670 tables and 37 types of formula functions. We realize the NL2Formula task by providing a sequence-to-sequence baseline implementation called fCoder. Experimental results validate the effectiveness of fCoder, demonstrating its superior performance compared to the baseline models. Furthermore, we also compare fCoder with an initial GPT-3.5 model (i.e., text-davinci-003). Lastly, through in-depth error analysis, we identify potential challenges in the NL2Formula task and advocate for further investigation.

Related papers

• SpreadsheetLLM: Encoding Spreadsheets for Large Language Models [44.08092362611575]
  SpreadsheetLLM is an efficient encoding method designed to unleash and optimize large language models (LLMs) on spreadsheets. We develop SheetCompressor, an innovative encoding framework that compresses spreadsheets effectively for LLMs. Fine-tuned LLM with SheetCompressor has an average compression ratio of 25 times, but achieves a state-of-the-art 78.9% F1 score, surpassing the best existing models by 12.3%.
  arXiv  Detail & Related papers  (2024-07-12T06:34:21Z)
• Auto-Formula: Recommend Formulas in Spreadsheets using Contrastive Learning for Table Representations [36.2969566996675]
  We develop an Auto-Formula system that can accurately predict formulas that users want to author in a target spreadsheet cell. We use contrastive-learning techniques inspired by "similar-face recognition" from compute vision.
  arXiv  Detail & Related papers  (2024-04-19T03:28:18Z)
• TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios [52.73289223176475]
  TableLLM is a robust large language model (LLM) with 13 billion parameters. TableLLM is purpose-built for proficiently handling data manipulation tasks. We have released the model checkpoint, source code, benchmarks, and a web application for user interaction.
  arXiv  Detail & Related papers  (2024-03-28T11:21:12Z)
• InstructExcel: A Benchmark for Natural Language Instruction in Excel [72.018640505825]
  This work investigates whether Large Language Models can generate code that solves Excel specific tasks provided via natural language user instructions. Our benchmark includes over 10k samples covering 170+ Excel operations across 2,000 publicly available Excel spreadsheets. We observe that (1) using GPT-4 over GPT-3.5, (2) providing more in-context examples, and (3) dynamic prompting can help improve performance on this benchmark.
  arXiv  Detail & Related papers  (2023-10-23T02:00:55Z)
• SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models [60.171444066848856]
  We propose a SheetCopilot agent that takes natural language task and control spreadsheet to fulfill the requirements. We curate a representative dataset containing 221 spreadsheet control tasks and establish a fully automated evaluation pipeline. Our SheetCopilot correctly completes 44.3% of tasks for a single generation, outperforming the strong code generation baseline by a wide margin.
  arXiv  Detail & Related papers  (2023-05-30T17:59:30Z)
• QTSumm: Query-Focused Summarization over Tabular Data [58.62152746690958]
  People primarily consult tables to conduct data analysis or answer specific questions. We define a new query-focused table summarization task, where text generation models have to perform human-like reasoning. We introduce a new benchmark named QTSumm for this task, which contains 7,111 human-annotated query-summary pairs over 2,934 tables.
  arXiv  Detail & Related papers  (2023-05-23T17:43:51Z)
• FLAME: A small language model for spreadsheet formulas [25.667479554632735]
  We present FLAME, a transformer-based model trained exclusively on Excel formulas. We use sketch deduplication, introduce an Excel-specific formula tokenizer, and use domain-specific versions of masked span prediction. We evaluate FLAME on formula repair, formula completion, and similarity-based formula retrieval.
  arXiv  Detail & Related papers  (2023-01-31T17:29:43Z)
• FORTAP: Using Formulae for Numerical-Reasoning-Aware Table Pretraining [23.747119682226675]
  FORTAP is the first method for numerical-reasoning-aware table pretraining by leveraging large corpus of spreadsheet formulae. FORTAP achieves results on two representative downstream tasks, cell type classification and formula prediction.
  arXiv  Detail & Related papers  (2021-09-15T14:31:17Z)
• SpreadsheetCoder: Formula Prediction from Semi-structured Context [70.41579328458116]
  We propose a BERT-based model architecture to represent the tabular context in both row-based and column-based formats. We train our model on a large dataset of spreadsheets, and demonstrate that SpreadsheetCoder achieves top-1 prediction accuracy of 42.51%. Compared to the rule-based system, SpreadsheetCoder 82% assists more users in composing formulas on Google Sheets.
  arXiv  Detail & Related papers  (2021-06-26T11:26:27Z)
• Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training [86.91380874390778]
  We present Generation-Augmented Pre-training (GAP), that jointly learns representations of natural language utterances and table schemas by leveraging generation models to generate pre-train data. Based on experimental results, neural semantics that leverage GAP MODEL obtain new state-of-the-art results on both SPIDER and CRITERIA-TO-generative benchmarks.
  arXiv  Detail & Related papers  (2020-12-18T15:53:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.