Structsum Generation for Faster Text Comprehension

• URL: http://arxiv.org/abs/2401.06837v2
• Date: Wed, 19 Jun 2024 09:59:51 GMT
• Title: Structsum Generation for Faster Text Comprehension
• Abstract summary: We consider the task of generating structured representations of text using large language models (LLMs) We focus on tables and mind maps as representative modalities. We show that current models struggle with generating structured outputs.
• Score: 5.708842985809019
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the task of generating structured representations of text using large language models (LLMs). We focus on tables and mind maps as representative modalities. Tables are more organized way of representing data, while mind maps provide a visually dynamic and flexible approach, particularly suitable for sparse content. Despite the effectiveness of LLMs on different tasks, we show that current models struggle with generating structured outputs. In response, we present effective prompting strategies for both of these tasks. We introduce a taxonomy of problems around factuality, global and local structure, common to both modalities and propose a set of critiques to tackle these issues resulting in an absolute improvement in accuracy of +37pp (79%) for mind maps and +15pp (78%) for tables. To evaluate semantic coverage of generated structured representations we propose Auto-QA, and we verify the adequacy of Auto-QA using SQuAD dataset. We further evaluate the usefulness of structured representations via a text comprehension user study. The results show a significant reduction in comprehension time compared to text when using table (42.9%) and mind map (31.9%), without loss in accuracy.

Related papers

• Monte Carlo Tree Search for Table-to-Multimodal Report Generation [15.410824932061011]
  This paper proposes a Monte Carlo Tree Search (MCTS)-driven framework that formulates multimodal table-to-report generation as a progressive construction process over a structured search space.<n>We use an LLM to generate step-by-step reasoning and actions during MCTS, storing the reasoning trajectory in each node for context-aware, coherent report construction.<n>Experiments on MMRBench demonstrate that MCTS-Report significantly outperforms strong baselines across structural completeness, numerical accuracy, chart-text alignment, and insight novelty, achieving a 77.9 overall score.
  arXiv  Detail & Related papers  (2026-08-04T15:52:05Z)
• TABVERSE: Benchmarking Cross-Format Table Understanding in LLMs and VLMs [50.96120467118319]
  Large Language Models (LLMs) and Vision-Language Models (VLMs) are increasingly evaluated on table reasoning tasks.<n>We introduce TABVERSE, a controlled multimodal table benchmark that aligns the same table content across multiple structural formats.<n>Our results show that representation choice substantially affects table understanding.
  arXiv  Detail & Related papers  (2026-06-08T14:52:46Z)
• OmniStruct: Universal Text-to-Structure Generation across Diverse Schemas [57.49565459553627]
  We introduce OmniStruct, a benchmark for assessing Large Language Models' capabilities on text-to-structure tasks.<n>We collect high-quality training data via synthetic task generation to facilitate the development of efficient text-to-structure models.<n>Our experiments demonstrate the possibility of fine-tuning much smaller models on synthetic data into universal structured generation models.
  arXiv  Detail & Related papers  (2025-11-23T08:18:12Z)
• CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning [67.18702329644526]
  CoT Referring enhances model reasoning across modalities through a structured, chain-of-thought training data structure.<n>We restructure the training data to enforce a new output form, providing new annotations for existing datasets.<n>We also integrate detection and segmentation capabilities into a unified MLLM framework, training it with a novel adaptive weighted loss to optimize performance.
  arXiv  Detail & Related papers  (2025-10-03T08:50:21Z)
• Same Content, Different Representations: A Controlled Study for Table QA [15.896655757672441]
  Table Question Answering (Table QA) in real-world settings must operate over both structured databases and semi-structured tables containing textual fields.<n>Existing benchmarks are tied to fixed data formats and have not systematically examined how representation itself affects model performance.<n>We present the first controlled study that isolates the role of table representation by holding content constant while varying structure.
  arXiv  Detail & Related papers  (2025-09-26T22:33:19Z)
• LLM driven Text-to-Table Generation through Sub-Tasks Guidance and Iterative Refinement [1.373677542041849]
  This paper proposes an efficient system for Large Language Models (LLMs)-driven text-to-table generation that leverages novel prompting techniques.<n>We show that this custom task decomposition allows the model to address the problem in a stepwise manner and improves the quality of the generated table.<n>Our methods achieve strong results compared to baselines on two complex text-to-table generation datasets available in the public domain.
  arXiv  Detail & Related papers  (2025-08-12T05:37:12Z)
• StructText: A Synthetic Table-to-Text Approach for Benchmark Generation with Multi-Dimensional Evaluation [8.251302684712773]
  StructText is an end-to-end framework for automatically generating high-fidelity benchmarks for key-value extraction from text.<n>We evaluate the proposed method on 71,539 examples across 49 documents.
  arXiv  Detail & Related papers  (2025-07-28T21:20:44Z)
• Map&Make: Schema Guided Text to Table Generation [41.52038779169547]
  Text-to-Table generation is an essential task for information retrieval.<n>We introduce a versatile approach, Map&Make, which "dissects" text into propositional atomic statements.<n>Our approach is tested against two challenging datasets, Rotowire and Livesum.
  arXiv  Detail & Related papers  (2025-05-29T07:12:46Z)
• Structuring the Unstructured: A Multi-Agent System for Extracting and Querying Financial KPIs and Guidance [54.25184684077833]
  We propose an efficient and scalable method for extracting quantitative insights from unstructured financial documents.<n>Our proposed system consists of two specialized agents: the emphExtraction Agent and the emphText-to-Agent
  arXiv  Detail & Related papers  (2025-05-25T15:45:46Z)
• Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflict [3.9495397165148023]
  We show how vision-language models (VLMs) reason when confronted with scenarios that present conflicting image and text cues. Our analysis shows that VLMs favor text in simpler queries but shift toward images as query complexity increases. In addition, we explore three mitigation strategies: simple prompt modifications, modifications that explicitly instruct models on how to handle conflicting information.
  arXiv  Detail & Related papers  (2025-04-11T20:56:52Z)
• Better Think with Tables: Tabular Structures Enhance LLM Comprehension for Data-Analytics Requests [33.471112091886894]
  Large Language Models (LLMs) often struggle with data-analytics requests related to information retrieval and data manipulation.<n>We introduce Thinking with Tables, where we inject tabular structures into LLMs for data-analytics requests.<n>We show that providing tables yields a 40.29 percent average performance gain along with better manipulation and token efficiency.
  arXiv  Detail & Related papers  (2024-12-22T23:31:03Z)
• Knowledge in Triples for LLMs: Enhancing Table QA Accuracy with Semantic Extraction [1.0968343822308813]
  This paper proposes a novel approach that extracts triples straightforward from tabular data and integrates it with a retrieval-augmented generation (RAG) model to enhance the accuracy, coherence, and contextual richness of responses generated by a fine-tuned GPT-3.5-turbo-0125 model. Our approach significantly outperforms existing baselines on the FeTaQA dataset, particularly excelling in Sacre-BLEU and ROUGE metrics.
  arXiv  Detail & Related papers  (2024-09-21T16:46:15Z)
• Knowledge-Aware Reasoning over Multimodal Semi-structured Tables [85.24395216111462]
  This study investigates whether current AI models can perform knowledge-aware reasoning on multimodal structured data. We introduce MMTabQA, a new dataset designed for this purpose. Our experiments highlight substantial challenges for current AI models in effectively integrating and interpreting multiple text and image inputs.
  arXiv  Detail & Related papers  (2024-08-25T15:17:43Z)
• On the Robustness of Language Models for Tabular Question Answering [7.486549276995143]
  Large Language Models (LLMs) have been shown to tackle table comprehension tasks without specific training. We evaluate the robustness of LLMs on Wikipedia-based textbfWTQ, financial report-based textbfTAT-QA, and scientific claims-based textbfSCITAB, TQA datasets.
  arXiv  Detail & Related papers  (2024-06-18T15:41:15Z)
• Facts-and-Feelings: Capturing both Objectivity and Subjectivity in Table-to-Text Generation [41.09752906121257]
  We introduce the Ta2TS dataset with 3849 data instances. We perform the task of fine-tuning sequence-to-sequence models on the linearized tables and prompting on popular large language models.
  arXiv  Detail & Related papers  (2024-06-15T08:41:44Z)
• TACT: Advancing Complex Aggregative Reasoning with Information Extraction Tools [51.576974932743596]
  Large Language Models (LLMs) often do not perform well on queries that require the aggregation of information across texts. TACT contains challenging instructions that demand stitching information scattered across one or more texts. We construct this dataset by leveraging an existing dataset of texts and their associated tables. We demonstrate that all contemporary LLMs perform poorly on this dataset, achieving an accuracy below 38%.
  arXiv  Detail & Related papers  (2024-06-05T20:32:56Z)
• Doc2SoarGraph: Discrete Reasoning over Visually-Rich Table-Text Documents via Semantic-Oriented Hierarchical Graphs [79.0426838808629]
  We propose TAT-DQA, i.e. to answer the question over a visually-rich table-text document. Specifically, we propose a novel Doc2SoarGraph framework with enhanced discrete reasoning capability. We conduct extensive experiments on TAT-DQA dataset, and the results show that our proposed framework outperforms the best baseline model by 17.73% and 16.91% in terms of Exact Match (EM) and F1 score respectively on the test set.
  arXiv  Detail & Related papers  (2023-05-03T07:30:32Z)
• Table Retrieval May Not Necessitate Table-specific Model Design [83.27735758203089]
  We focus on the task of table retrieval, and ask: "is table-specific model design necessary for table retrieval?" Based on an analysis on a table-based portion of the Natural Questions dataset (NQ-table), we find that structure plays a negligible role in more than 70% of the cases. We then experiment with three modules to explicitly encode table structures, namely auxiliary row/column embeddings, hard attention masks, and soft relation-based attention biases. None of these yielded significant improvements, suggesting that table-specific model design may not be necessary for table retrieval.
  arXiv  Detail & Related papers  (2022-05-19T20:35:23Z)
• Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsing [52.24507547010127]
  Cross-domain context-dependent semantic parsing is a new focus of research. We present a dynamic graph framework that effectively modelling contextual utterances, tokens, database schemas, and their complicated interaction as the conversation proceeds. The proposed framework outperforms all existing models by large margins, achieving new state-of-the-art performance on two large-scale benchmarks.
  arXiv  Detail & Related papers  (2021-01-05T18:11:29Z)
• Automated Concatenation of Embeddings for Structured Prediction [75.44925576268052]
  We propose Automated Concatenation of Embeddings (ACE) to automate the process of finding better concatenations of embeddings for structured prediction tasks. We follow strategies in reinforcement learning to optimize the parameters of the controller and compute the reward based on the accuracy of a task model.
  arXiv  Detail & Related papers  (2020-10-10T14:03:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.