LLM StructCore: Schema-Guided Reasoning Condensation and Deterministic Compilation

• URL: http://arxiv.org/abs/2604.20560v1
• Date: Wed, 22 Apr 2026 13:42:00 GMT
• Title: LLM StructCore: Schema-Guided Reasoning Condensation and Deterministic Compilation
• Abstract summary: We describe our CL4Health 2026 submission for Dyspnea CRF filling (134 items) using a contract-driven two-stage design grounded in Contracts-Guided Reasoning (SGR)<n>The key task property is extreme sparsity: the majority of fields are unknown, and official scoring penalizes both empty values and unsupported predictions.<n>We shift from a single-step "LLM" approach to a fully deterministic, 0-LLM compiler that parses Stage 1 summary, normalizes item names, applies evidence-gated false-positive filters, and expands the output into the required 134-item format.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatically filling Case Report Forms (CRFs) from clinical notes is challenging due to noisy language, strict output contracts, and the high cost of false positives. We describe our CL4Health 2026 submission for Dyspnea CRF filling (134 items) using a contract-driven two-stage design grounded in Schema-Guided Reasoning (SGR). The key task property is extreme sparsity: the majority of fields are unknown, and official scoring penalizes both empty values and unsupported predictions. We shift from a single-step "LLM predicts 134 fields" approach to a decomposition where (i) Stage 1 produces a stable SGR-style JSON summary with exactly 9 domain keys, and (ii) Stage 2 is a fully deterministic, 0-LLM compiler that parses the Stage 1 summary, canonicalizes item names, normalizes predictions to the official controlled vocabulary, applies evidence-gated false-positive filters, and expands the output into the required 134-item format. On the dev80 split, the best teacher configuration achieves macro-F1 0.6543 (EN) and 0.6905 (IT); on the hidden test200, the submitted English variant scores 0.63 on Codabench. The pipeline is language-agnostic: Italian results match or exceed English with no language-specific engineering.

Related papers

• PolicyKG: An Agentic LLM Pipeline for Translating Institutional Policies into SHACL Knowledge Graphs [2.4851820343103035]
  PolicyKG reads a policy PDF, classifies each sentence as an obligation, permission, or prohibition.<n>It then lifts the label into first-order deontic logic, and emits SHACL constraints.<n>Four stages run on a LangGraph state machine with per-stage validators.
  arXiv  Detail & Related papers  (2026-08-10T02:28:57Z)
• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• UOL@IDEM at BEA 2026 Shared Task 1: Neural Fusion and Feature-Rich Modeling for L1-Aware Vocabulary Difficulty Prediction [1.9746060146273674]
  This paper describes UOL@IDEM's closed-track submission to the BEA 2026 shared task on L1-aware vocabulary difficulty prediction.<n>We model the task as regression and train separate systems for Spanish, German, and Mandarin Chinese.<n>Our system combines multilingual contextual representations with engineered features capturing frequency, surface form, retrieval evidence, semantic alignment, cognate similarity, and masked-language-model predictability.
  arXiv  Detail & Related papers  (2026-06-23T12:30:48Z)
• Context-Aware Distillation and Ablation for Text2DSL [78.29352305480285]
  We replace prompt-only synthetic generation with context-aware distillation.<n>A teacher large language model operates under an explicitly defined structured context.<n>This scales the verified PolkitBench corpus from 4,204 to 10,073 pairs.
  arXiv  Detail & Related papers  (2026-06-21T16:27:24Z)
• Benchmarking Speech-to-Speech Translation Models [55.00303727199927]
  Speech-to-speech translation (S2ST) has advanced rapidly, but offline evaluation lacks a unified protocol.<n>We introduce a benchmarking framework integrating 46 metrics across eight dimensions.<n>We deploy it on 1,248 model-language configurations from FLEURS and CVSS.
  arXiv  Detail & Related papers  (2026-06-02T07:01:33Z)
• LLMEval-Logic: A Solver-Verified Chinese Benchmark for Logical Reasoning of LLMs with Adversarial Hardening [69.1037790901185]
  We present LLMEval-Logic, a Chinese logical reasoning benchmark built from realistic situational scenarios.<n>Its pipeline forward-authors and expert-audits natural-language items together with their reference formalizations, verifies annotated answers with Z3, constructs expert rubrics for natural-to-formal grading, and hardens selected items through a closed-loop adversarial workflow.
  arXiv  Detail & Related papers  (2026-05-19T09:40:29Z)
• SlovKE: A Large-Scale Dataset and LLM Evaluation for Slovak Keyphrase Extraction [0.00954904463032233]
  Keyphrase extraction for morphologically rich, low-resource languages remains understudied.<n>We construct a dataset of 227,432 scientific abstracts with author-assigned keyphrases.<n>KeyLLM produces keyphrases closer to the canonical forms assigned by authors.<n>Our analysis identifies morphological mismatch as the dominant failure mode for statistical methods.
  arXiv  Detail & Related papers  (2026-03-16T16:47:45Z)
• OpenSanctions Pairs: Large-Scale Entity Matching with LLMs [0.9131359219276399]
  We release OpenSanctions Pairs, a large-scale entity matching benchmark derived from real-world international sanctions aggregation and analyst deduplication.<n>The dataset contains 755,540 labeled pairs spanning 293 heterogeneous sources across 31 countries.<n>Off-the-shelf LLMs substantially outperform the production rule-based baseline.
  arXiv  Detail & Related papers  (2026-02-24T06:25:49Z)
• LLMSR@XLLM25: An Empirical Study of LLM for Structural Reasoning [6.700515856842664]
  We present Team asdfo123's submission to the LLMSR@XLLM25 shared task.<n>We evaluate large language models on producing fine-grained, controllable, and interpretable reasoning processes.<n>Our method ranks 5th overall, achieving macro F1 scores on par with substantially more complex and resource-consuming pipelines.
  arXiv  Detail & Related papers  (2025-05-18T09:46:30Z)
• Paloma: A Benchmark for Evaluating Language Model Fit [112.481957296585]
  Evaluations of language models (LMs) commonly report perplexity on monolithic data held out from training.<n>We introduce Perplexity Analysis for Language Model Assessment (Paloma), a benchmark to measure LM fit to 546 English and code domains.
  arXiv  Detail & Related papers  (2023-12-16T19:12:45Z)
• Evaluating the Factual Consistency of Large Language Models Through News Summarization [97.04685401448499]
  We propose a new benchmark called FIB(Factual Inconsistency Benchmark) that focuses on the task of summarization. For factually consistent summaries, we use human-written reference summaries that we manually verify as factually consistent. For factually inconsistent summaries, we generate summaries from a suite of summarization models that we have manually annotated as factually inconsistent.
  arXiv  Detail & Related papers  (2022-11-15T18:50:34Z)
• CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation [113.99145386490639]
  Cross-lingual NER can transfer knowledge between languages via aligned cross-lingual representations or machine translation results. We propose a Cross-lingual Entity Projection framework (CROP) to enable zero-shot cross-lingual NER. We adopt a multilingual labeled sequence translation model to project the tagged sequence back to the target language and label the target raw sentence.
  arXiv  Detail & Related papers  (2022-10-13T13:32:36Z)
• IDIAPers @ Causal News Corpus 2022: Efficient Causal Relation Identification Through a Prompt-based Few-shot Approach [3.4423596432619754]
  We address the Causal Relation Identification (CRI) task by exploiting a set of simple yet complementary techniques for fine-tuning language models (LMs) We follow a prompt-based prediction approach for fine-tuning LMs in which the CRI task is treated as a masked language modeling problem (MLM) We compare the performance of this method against ensemble techniques trained on the entire dataset.
  arXiv  Detail & Related papers  (2022-09-08T16:03:50Z)
• Towards No.1 in CLUE Semantic Matching Challenge: Pre-trained Language Model Erlangshen with Propensity-Corrected Loss [12.034243662298035]
  This report describes a pre-trained language model Erlangshen with propensity-corrected loss. We construct a dynamic masking strategy based on knowledge in Masked Language Modeling (MLM) with whole word masking. Overall, we achieve 72.54 points in F1 Score and 78.90 points in Accuracy on the test set.
  arXiv  Detail & Related papers  (2022-08-05T02:52:29Z)
• LM-Critic: Language Models for Unsupervised Grammatical Error Correction [128.9174409251852]
  We show how to leverage a pretrained language model (LM) in defining an LM-Critic, which judges a sentence to be grammatical. We apply this LM-Critic and BIFI along with a large set of unlabeled sentences to bootstrap realistic ungrammatical / grammatical pairs for training a corrector.
  arXiv  Detail & Related papers  (2021-09-14T17:06:43Z)
• Explicit Alignment Objectives for Multilingual Bidirectional Encoders [111.65322283420805]
  We present a new method for learning multilingual encoders, AMBER (Aligned Multilingual Bi-directional EncodeR) AMBER is trained on additional parallel data using two explicit alignment objectives that align the multilingual representations at different granularities. Experimental results show that AMBER obtains gains of up to 1.1 average F1 score on sequence tagging and up to 27.3 average accuracy on retrieval over the XLMR-large model.
  arXiv  Detail & Related papers  (2020-10-15T18:34:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.