Where vs What: Decomposing Structural and Content Failures in LLM-Generated Structured Outputs

• URL: http://arxiv.org/abs/2608.25358v1
• Date: Wed, 26 Aug 2026 04:32:06 GMT
• Title: Where vs What: Decomposing Structural and Content Failures in LLM-Generated Structured Outputs
• Abstract summary: We introduce Structure-Content Decomposition (SCD), a framework that measures structural fidelity and content accuracy.<n>Applying SCD to nested and table tasks across six models (7B to frontier), we uncover a consistent phenomenon: structural fidelity degrades earlier and more sharply than content accuracy increases.<n>We propose SA-RLVR, converting SCD metrics into verifiable rewards for learning.
• Score: 15.823269213947347
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Structured outputs such as JSON and tables are central to modern LLM-based systems, yet generation failures are evaluated monolithically, conflating two distinct error modes: placement errors (correct values at wrong positions) and value errors (wrong values at intended positions). We introduce Structure-Content Decomposition (SCD), a framework that independently measures structural fidelity and content accuracy. Applying SCD to nested JSON and table tasks across six models (7B to frontier), we uncover a consistent phenomenon: structural fidelity degrades earlier and more sharply than content accuracy as complexity increases. At the highest complexity, even DeepSeek-V4-Flash (with reasoning) misplaces 35% of recalled values, while Qwen2.5-7B misplaces 74%. Controlled ablations suggest that this pattern is associated with reliance on semantic shortcuts rather than topological understanding of output structure. Based on these findings, we propose SA-RLVR, converting SCD metrics into verifiable rewards for reinforcement learning via GRPO. SA-RLVR successfully optimizes structural addressing across distinct topologies: it lifts JSON Value Placement Accuracy (VPA) from 26% to 63% while generalizing to held-out schemas; moreover, it consistently drives VPA improvements in the table domain, demonstrating that structure-aware rewards can directly enhance multi-domain structural positioning.

Related papers

• Alignment-Guided Largest Table Overlap Size Estimation [11.275866569459502]
  Fast estimation of the size of the largest overlap between tables enables blocking and query-by-table retrieval in large table repositories.<n>We propose ALORE, a scalable and domain-robust overlap ratio estimator built on three principles.<n> Experiments on multiple datasets spanning diverse domains and scales, including a large real-world corpus beyond prior benchmarks, show that ALORE outperforms the state of the art.
  arXiv  Detail & Related papers  (2026-07-03T07:38:43Z)
• TS-Fault: Benchmarking Time Series Forecasters Against Structural Faults [22.88447833683736]
  Time series forecasting (TSF) underpins consequential decisions in energy, transportation, finance, and healthcare.<n>TSF models are almost universally ranked by a single number on clean held-out data, under the implicit assumption that it predicts deployed reliability.<n>We present TS-Fault, a benchmark that evaluates forecasting models under explicit, parameterized fault scenarios with controllable semantic difficulty.
  arXiv  Detail & Related papers  (2026-06-16T23:15:20Z)
• APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay [7.370176470430802]
  LLM-based autonomous agents lack persistent procedural memory.<n>We present APEX-EM, a non-parametric online learning framework that accumulates, retrieves, and reuses structured procedural plans.
  arXiv  Detail & Related papers  (2026-03-31T00:24:56Z)
• Architecture-Aware Multi-Design Generation for Repository-Level Feature Addition [53.50448142467294]
  RAIM is a multi-design and architecture-aware framework for repository-level feature addition.<n>It shifts away from linear patching by generating multiple diverse implementation designs.<n>Experiments on the NoCode-bench Verified dataset demonstrate that RAIM establishes a new state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-03-02T12:50:40Z)
• RL-Struct: A Lightweight Reinforcement Learning Framework for Reliable Structured Output in LLMs [0.08594140167290097]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language generation and reasoning.<n>Their integration into automated software ecosystems is often hindered by the "Structure Gap"<n>We propose a lightweight, efficient Reinforcement Learning framework to bridge this gap.
  arXiv  Detail & Related papers  (2025-11-29T04:47:14Z)
• STED and Consistency Scoring: A Framework for Evaluating LLM Structured Output Reliability [11.095198847819573]
  Large Language Models (LLMs) are increasingly deployed for structured data generation.<n>We introduce a comprehensive framework for evaluating and improving consistency in LLM-generated structured outputs.
  arXiv  Detail & Related papers  (2025-11-27T02:49:52Z)
• Self-Correction Distillation for Structured Data Question Answering [50.98882432829651]
  Small-scale language models (LLMs) are prone to errors in generating structured queries.<n>We propose a self-correction distillation (SCD) method to improve the structured data QA ability of small-scale LLMs.
  arXiv  Detail & Related papers  (2025-11-11T09:01:51Z)
• AlphaFold Database Debiasing for Robust Inverse Folding [58.792020809180336]
  We introduce a Debiasing Structure AutoEncoder (DeSAE) that learns to reconstruct native-like conformations from intentionally corrupted backbone geometries.<n>At inference, applying DeSAE to AFDB structures produces debiased structures that significantly improve inverse folding performance.
  arXiv  Detail & Related papers  (2025-06-10T02:25:31Z)
• Elucidating the Design Space of Multimodal Protein Language Models [69.3650883370033]
  Multimodal protein language models (PLMs) integrate sequence and token-based structural information.<n>This paper systematically elucidates the design space of multimodal PLMs to overcome their limitations.<n>Our advancements approach finer-grained supervision, demonstrating that token-based multimodal PLMs can achieve robust structural modeling.
  arXiv  Detail & Related papers  (2025-04-15T17:59:43Z)
• Robust Self-Supervised LiDAR Odometry via Representative Structure Discovery and 3D Inherent Error Modeling [67.75095378830694]
  We develop a two-stage odometry estimation network, where we obtain the ego-motion by estimating a set of sub-region transformations. In this paper, we aim to alleviate the influence of unreliable structures in training, inference and mapping phases. Our two-frame odometry outperforms the previous state of the arts by 16%/12% in terms of translational/rotational errors.
  arXiv  Detail & Related papers  (2022-02-27T12:52:27Z)
• ISTR: End-to-End Instance Segmentation with Transformers [147.14073165997846]
  We propose an instance segmentation Transformer, termed ISTR, which is the first end-to-end framework of its kind. ISTR predicts low-dimensional mask embeddings, and matches them with ground truth mask embeddings for the set loss. Benefiting from the proposed end-to-end mechanism, ISTR demonstrates state-of-the-art performance even with approximation-based suboptimal embeddings.
  arXiv  Detail & Related papers  (2021-05-03T06:00:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.