OmniStruct: Universal Text-to-Structure Generation across Diverse Schemas

• URL: http://arxiv.org/abs/2511.18335v1
• Date: Sun, 23 Nov 2025 08:18:12 GMT
• Title: OmniStruct: Universal Text-to-Structure Generation across Diverse Schemas
• Authors: James Y. Huang, Wenxuan Zhou, Nan Xu, Fei Wang, Qin Liu, Sheng Zhang, Hoifung Poon, Muhao Chen,
• Abstract summary: We introduce OmniStruct, a benchmark for assessing Large Language Models' capabilities on text-to-structure tasks.<n>We collect high-quality training data via synthetic task generation to facilitate the development of efficient text-to-structure models.<n>Our experiments demonstrate the possibility of fine-tuning much smaller models on synthetic data into universal structured generation models.
• Score: 57.49565459553627
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ability of Large Language Models (LLMs) to generate structured outputs that follow arbitrary schemas is crucial to a wide range of downstream tasks that require diverse structured representations of results such as information extraction, table generation, and function calling. While modern LLMs excel in generating unstructured responses in natural language, whether this advancement translates to a strong performance on text-to-structure tasks remains unclear. To bridge this gap, we first introduce OmniStruct, a comprehensive benchmark for assessing LLMs' capabilities on diverse text-to-structure tasks such as information extraction, table generation, and function calling. We build OmniStruct by identifying existing datasets across a wide range of tasks that are suitable for a structured answer format, and adapting them under a unified text-to-structure problem setting. To facilitate the development of efficient text-to-structure models, we collect high-quality training data via synthetic task generation. Without using any supervised data for OmniStruct tasks, our experiments demonstrate the possibility of fine-tuning much smaller models on synthetic data into universal structured generation models that can rival the performance of GPT-4o.

Related papers

• LLM driven Text-to-Table Generation through Sub-Tasks Guidance and Iterative Refinement [1.373677542041849]
  This paper proposes an efficient system for Large Language Models (LLMs)-driven text-to-table generation that leverages novel prompting techniques.<n>We show that this custom task decomposition allows the model to address the problem in a stepwise manner and improves the quality of the generated table.<n>Our methods achieve strong results compared to baselines on two complex text-to-table generation datasets available in the public domain.
  arXiv  Detail & Related papers  (2025-08-12T05:37:12Z)
• Beyond Isolated Dots: Benchmarking Structured Table Construction as Deep Knowledge Extraction [80.88654868264645]
  Arranged and Organized Extraction Benchmark designed to evaluate ability of large language models to comprehend fragmented documents.<n>AOE includes 11 carefully crafted tasks across three diverse domains, requiring models to generate context-specific schema tailored to varied input queries.<n>Results show that even the most advanced models struggled significantly.
  arXiv  Detail & Related papers  (2025-07-22T06:37:51Z)
• StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputs [39.108050455592036]
  StructEval is a benchmark for evaluating Large Language Models' capabilities in producing structured formats.<n>Our benchmark encompasses 18 formats and 44 types of task, with novel metrics for format adherence and structural correctness.<n>Results reveal significant performance gaps, even state-of-the-art models like o1-mini achieve only 75.58 average score.
  arXiv  Detail & Related papers  (2025-05-26T15:40:42Z)
• The Effectiveness of Large Language Models in Transforming Unstructured Text to Standardized Formats [0.0]
  This study systematically evaluating Large Language Models' ability to convert unstructured text into structured formats.<n>Experiments reveal that GPT-4o with few-shot prompting achieves breakthrough performance.<n>These findings open new possibilities for automated structured data generation across various domains.
  arXiv  Detail & Related papers  (2025-03-04T14:14:28Z)
• Struct-X: Enhancing Large Language Models Reasoning with Structured Data [38.558614152006975]
  Struct-X operates through five key phases: read-model-fill-reflect-reason'' It encodes structured data into a topological space using graph embeddings. It fills in missing entity information with knowledge retrieval modules. The final phase involves constructing a topological network with selected tokens.
  arXiv  Detail & Related papers  (2024-07-17T13:06:25Z)
• StrucText-Eval: Evaluating Large Language Model's Reasoning Ability in Structure-Rich Text [29.03935605732864]
  We introduce StrucText-Eval, a benchmark to evaluate how well large language models understand and reason through structured text. We show that while open-source LLMs achieve a maximum accuracy of 74.9% on the standard dataset, their performance drops significantly to 45.8% on the harder dataset. In contrast, human participants reach an accuracy of 92.6% on StrucText-Eval-Hard, highlighting LLMs' current limitations in handling intricate structural information.
  arXiv  Detail & Related papers  (2024-06-15T12:48:00Z)
• StructLM: Towards Building Generalist Models for Structured Knowledge Grounding [49.10029030628653]
  Large language models' (LLMs) ability to process structured data lags behind state-of-the-art (SoTA) model by an average of 35%. We train a series of models, referred to as StructLM, based on the Mistral and the CodeLlama model family, ranging from 7B to 34B parameters. Our StructLM series surpasses task-specific models on 16 out of 18 evaluated datasets and establishes new SoTA performance on 8 SKG tasks.
  arXiv  Detail & Related papers  (2024-02-26T15:47:01Z)
• A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction [11.165093163378152]
  Large language models (LLMs) have demonstrated impressive abilities in generating unstructured natural language according to instructions. This paper introduces an efficient method, G&O, to enhance their structured text generation capabilities.
  arXiv  Detail & Related papers  (2024-02-20T20:42:02Z)
• Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data? [49.688233418425995]
  Struc-Bench is a comprehensive benchmark featuring prominent Large Language Models (LLMs) We propose two innovative metrics, P-Score (Prompting Score) and H-Score (Heuristical Score) Our experiments show that applying our structure-aware fine-tuning to LLaMA-7B leads to substantial performance gains.
  arXiv  Detail & Related papers  (2023-09-16T11:31:58Z)
• StructGPT: A General Framework for Large Language Model to Reason over Structured Data [117.13986738340027]
  We develop an emphIterative Reading-then-Reasoning(IRR) approach for solving question answering tasks based on structured data. Our approach can significantly boost the performance of ChatGPT and achieve comparable performance against the full-data supervised-tuning baselines.
  arXiv  Detail & Related papers  (2023-05-16T17:45:23Z)
• Unified Structure Generation for Universal Information Extraction [58.89057387608414]
  UIE can universally model different IE tasks, adaptively generate targeted structures, and collaboratively learn general IE abilities from different knowledge sources. Experiments show that UIE achieved the state-of-the-art performance on 4 IE tasks, 13 datasets, and on all supervised, low-resource, and few-shot settings.
  arXiv  Detail & Related papers  (2022-03-23T08:49:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.