Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction

• URL: http://arxiv.org/abs/2404.14215v1
• Date: Mon, 22 Apr 2024 14:31:28 GMT
• Title: Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction
• Authors: Zheye Deng, Chunkit Chan, Weiqi Wang, Yuxi Sun, Wei Fan, Tianshi Zheng, Yauwai Yim, Yangqiu Song,
• Abstract summary: We introduce LiveSum, a new benchmark dataset for generating summary tables of competitions based on real-time commentary texts. We evaluate the performances of state-of-the-art Large Language Models on this task in both fine-tuning and zero-shot settings. We additionally propose a novel pipeline called $T3$(Text-Tuple-Table) to improve their performances.
• Score: 36.915250638481986
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The task of condensing large chunks of textual information into concise and structured tables has gained attention recently due to the emergence of Large Language Models (LLMs) and their potential benefit for downstream tasks, such as text summarization and text mining. Previous approaches often generate tables that directly replicate information from the text, limiting their applicability in broader contexts, as text-to-table generation in real-life scenarios necessitates information extraction, reasoning, and integration. However, there is a lack of both datasets and methodologies towards this task. In this paper, we introduce LiveSum, a new benchmark dataset created for generating summary tables of competitions based on real-time commentary texts. We evaluate the performances of state-of-the-art LLMs on this task in both fine-tuning and zero-shot settings, and additionally propose a novel pipeline called $T^3$(Text-Tuple-Table) to improve their performances. Extensive experimental results demonstrate that LLMs still struggle with this task even after fine-tuning, while our approach can offer substantial performance gains without explicit training. Further analyses demonstrate that our method exhibits strong generalization abilities, surpassing previous approaches on several other text-to-table datasets. Our code and data can be found at https://github.com/HKUST-KnowComp/LiveSum-TTT.

Related papers

• RAGulator: Lightweight Out-of-Context Detectors for Grounded Text Generation [3.2234520582876467]
  We train lightweight models to discriminate LLM-generated text that is semantically out-of-context from retrieved text documents. We find that DeBERTa is not only the best-performing model under this pipeline, but it is also fast and does not require additional text preprocessing or feature engineering.
  arXiv  Detail & Related papers  (2024-11-06T13:51:42Z)
• Integrating Planning into Single-Turn Long-Form Text Generation [66.08871753377055]
  We propose to use planning to generate long form content. Our main novelty lies in a single auxiliary task that does not require multiple rounds of prompting or planning. Our experiments demonstrate on two datasets from different domains, that LLMs fine-tuned with the auxiliary task generate higher quality documents.
  arXiv  Detail & Related papers  (2024-10-08T17:02:40Z)
• Scaling Up Summarization: Leveraging Large Language Models for Long Text Extractive Summarization [0.27624021966289597]
  This paper introduces EYEGLAXS, a framework that leverages Large Language Models (LLMs) for extractive summarization. EYEGLAXS focuses on extractive summarization to ensure factual and grammatical integrity. The system sets new performance benchmarks on well-known datasets like PubMed and ArXiv.
  arXiv  Detail & Related papers  (2024-08-28T13:52:19Z)
• TACT: Advancing Complex Aggregative Reasoning with Information Extraction Tools [51.576974932743596]
  Large Language Models (LLMs) often do not perform well on queries that require the aggregation of information across texts. TACT contains challenging instructions that demand stitching information scattered across one or more texts. We construct this dataset by leveraging an existing dataset of texts and their associated tables. We demonstrate that all contemporary LLMs perform poorly on this dataset, achieving an accuracy below 38%.
  arXiv  Detail & Related papers  (2024-06-05T20:32:56Z)
• CT-Eval: Benchmarking Chinese Text-to-Table Performance in Large Language Models [36.82189550072201]
  Existing text-to-table datasets are typically oriented English. Large language models (LLMs) have shown great success as general task solvers in multi-lingual settings. We propose a Chinese text-to-table dataset, CT-Eval, to benchmark LLMs on this task.
  arXiv  Detail & Related papers  (2024-05-20T16:58:02Z)
• All Data on the Table: Novel Dataset and Benchmark for Cross-Modality Scientific Information Extraction [39.05577374775964]
  We propose a semi-supervised pipeline for annotating entities in text, as well as entities and relations in tables, in an iterative procedure. We release novel resources for the scientific community, including a high-quality benchmark, a large-scale corpus, and a semi-supervised annotation pipeline.
  arXiv  Detail & Related papers  (2023-11-14T14:22:47Z)
• Abstractive Text Summarization for Resumes With Cutting Edge NLP Transformers and LSTM [0.0]
  LSTM, pre-trained models, and fine-tuned models were assessed using a dataset of resumes. The BART-Large model fine-tuned with the resume dataset gave the best performance.
  arXiv  Detail & Related papers  (2023-06-23T06:33:20Z)
• Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning [51.90524745663737]
  A key innovation is our use of explanations as features, which can be used to boost GNN performance on downstream tasks. Our method achieves state-of-the-art results on well-established TAG datasets. Our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv.
  arXiv  Detail & Related papers  (2023-05-31T03:18:03Z)
• QTSumm: Query-Focused Summarization over Tabular Data [58.62152746690958]
  People primarily consult tables to conduct data analysis or answer specific questions. We define a new query-focused table summarization task, where text generation models have to perform human-like reasoning. We introduce a new benchmark named QTSumm for this task, which contains 7,111 human-annotated query-summary pairs over 2,934 tables.
  arXiv  Detail & Related papers  (2023-05-23T17:43:51Z)
• GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing [117.98107557103877]
  We present GraPPa, an effective pre-training approach for table semantic parsing. We construct synthetic question-pairs over high-free tables via a synchronous context-free grammar. To maintain the model's ability to represent real-world data, we also include masked language modeling.
  arXiv  Detail & Related papers  (2020-09-29T08:17:58Z)
• ToTTo: A Controlled Table-To-Text Generation Dataset [61.83159452483026]
  ToTTo is an open-domain English table-to-text dataset with over 120,000 training examples. We introduce a dataset construction process where annotators directly revise existing candidate sentences from Wikipedia. While usually fluent, existing methods often hallucinate phrases that are not supported by the table.
  arXiv  Detail & Related papers  (2020-04-29T17:53:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.