LLM-Guided Probabilistic Fusion for Label-Efficient Document Layout Analysis

• URL: http://arxiv.org/abs/2511.08903v2
• Date: Fri, 14 Nov 2025 01:32:34 GMT
• Title: LLM-Guided Probabilistic Fusion for Label-Efficient Document Layout Analysis
• Abstract summary: Document layout understanding remains data-intensive despite advances in semi-supervised learning.<n>We present a framework that enhances semi-supervised detection by fusing visual predictions with structural priors.<n>Our method demonstrates consistent gains across model scales.
• Score: 6.908972852063454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Document layout understanding remains data-intensive despite advances in semi-supervised learning. We present a framework that enhances semi-supervised detection by fusing visual predictions with structural priors from text-pretrained LLMs via principled probabilistic weighting. Given unlabeled documents, an OCR-LLM pipeline infers hierarchical regions which are combined with teacher detector outputs through inverse-variance fusion to generate refined pseudo-labels.Our method demonstrates consistent gains across model scales. With a lightweight SwiftFormer backbone (26M params), we achieve 88.2$\pm$0.3 AP using only 5\% labels on PubLayNet. When applied to document-pretrained LayoutLMv3 (133M params), our fusion framework reaches 89.7$\pm$0.4 AP, surpassing both LayoutLMv3 with standard semi-supervised learning (89.1$\pm$0.4 AP, p=0.02) and matching UDOP~\cite{udop} (89.8 AP) which requires 100M+ pages of multimodal pretraining. This demonstrates that LLM structural priors are complementary to both lightweight and pretrained architectures. Key findings include: (1) learned instance-adaptive gating improves over fixed weights by +0.9 AP with data-dependent PAC bounds correctly predicting convergence; (2) open-source LLMs enable privacy-preserving deployment with minimal loss (Llama-3-70B: 87.1 AP lightweight, 89.4 AP with LayoutLMv3); (3) LLMs provide targeted semantic disambiguation (18.7\% of cases, +3.8 AP gain) beyond simple text heuristics.Total system cost includes \$12 for GPT-4o-mini API or 17 GPU-hours for local Llama-3-70B per 50K pages, amortized across training runs.

Related papers

• Your Retriever Already Knows: Distribution-Shape QPP for RAG Retrieval Sufficiency [0.0]
  Motivated by a Czech nuclear-regulator deployment, we compare three Query Performance Prediction (QPP) paradigms for retrieval sufficiency in RAG.<n>Our 24 non-lexical features reach a weighted-average AUROC of 0.856 at 2 ms per query, ahead of a classic-QPP literature pool.<n>Adding the LLM judgment as one feature (hybrid) matches S1 on ViDoRe (0.863) but gains a statistically significant edge on SJB.
  arXiv  Detail & Related papers  (2026-09-10T14:52:16Z)
• AnchorScore: A CLIP-Based Diagnostic of MLLM Annotation Difficulty [7.105430294335961]
  Multimodal large language models (MLLMs) are widely used for automated annotation, yet their per-class accuracy varies widely.<n>This paper systematically evaluates its full-frame formulation, termed AnchorScore, that assesses the classes MLLMs are least likely to annotate reliably.
  arXiv  Detail & Related papers  (2026-08-17T15:10:33Z)
• LLM-Redactor: An Empirical Evaluation of Eight Techniques for Privacy-Preserving LLM Requests [0.0]
  Coding agents and LLM-powered applications routinely send potentially sensitive content to cloud LLM APIs where it may be logged, retained, used for training, or subpoenaed.<n>We present a systematic empirical evaluation of eight techniques for privacy-preserving LLM requests.<n>We implement all eight (or a tractable research-stage subset where deployment is not yet feasible) in an open-source shim compatible with MCP and any OpenAI-compatible API.
  arXiv  Detail & Related papers  (2026-04-13T21:05:42Z)
• A Reliability Evaluation of Hybrid Deterministic-LLM Based Approaches for Academic Course Registration PDF Information Extraction [0.0]
  This study evaluates the reliability of information extraction approaches from KRS documents using three strategies.<n>Experiments were conducted on 140 documents for the LLM based test and 860 documents for the Camelot based pipeline evaluation.
  arXiv  Detail & Related papers  (2026-03-09T02:09:02Z)
• A BERTology View of LLM Orchestrations: Token- and Layer-Selective Probes for Efficient Single-Pass Classification [2.0069888187253615]
  Production LLM systems often rely on separate models for safety and other classification-heavy steps.<n>We instead reuse computation already paid for by the serving LLM: we train lightweight probes on its hidden states and predict labels in the same forward pass used for generation.
  arXiv  Detail & Related papers  (2026-01-19T18:40:29Z)
• MI-Fuse: Label Fusion for Unsupervised Domain Adaptation with Closed-Source Large-Audio Language Model [49.59690207400984]
  Large audio-language models (LALMs) show strong zero-shot ability on speech tasks, suggesting promise for speech emotion recognition (SER)<n>We ask: given only unlabeled target-domain audio and an API-only LALM, can a student model be adapted to outperform the LALM in the target domain?<n>We propose MI-Fuse, a denoised label fusion framework that supplements the LALM with a source-domain trained SER as an auxiliary teacher.
  arXiv  Detail & Related papers  (2025-09-25T03:16:32Z)
• Memento: Fine-tuning LLM Agents without Fine-tuning LLMs [36.3424780932712]
  We introduce a novel learning paradigm for Adaptive Large Language Model (LLM) agents.<n>Our method enables low-cost continual adaptation via memory-based online reinforcement learning.<n>We instantiate our agent model in the deep research setting, namely emphMemento, which attains top-1 on GAIA validation.
  arXiv  Detail & Related papers  (2025-08-22T07:25:30Z)
• Open-Source LLMs Collaboration Beats Closed-Source LLMs: A Scalable Multi-Agent System [51.04535721779685]
  This paper aims to demonstrate the potential and strengths of open-source collectives.<n>We propose SMACS, a scalable multi-agent collaboration system (MACS) framework with high performance.<n> Experiments on eight mainstream benchmarks validate the effectiveness of our SMACS.
  arXiv  Detail & Related papers  (2025-07-14T16:17:11Z)
• How to Train Your LLM Web Agent: A Statistical Diagnosis [96.86317871461834]
  We present the first statistically grounded study on compute allocation for LLM web-agent post-training.<n>Our approach uses a two-stage pipeline, training a Llama 3.1 8B student to imitate a Llama 3.3 70B teacher via supervised fine-tuning (SFT) and on-policy reinforcement learning.<n>Our results show that combining SFT with on-policy RL consistently outperforms either approach alone on both WorkArena and MiniWob++.
  arXiv  Detail & Related papers  (2025-07-05T17:12:33Z)
• Evaluating the Use of LLMs for Documentation to Code Traceability [3.076436880934678]
  Large Language Models can establish trace links between various software documentation and source code.<n>We create two novel datasets from two open-source projects (Unity Catalog and Crawl4AI)<n>Results show that the best-performing LLM achieves F1-scores of 79.4% and 80.4% across the two datasets.
  arXiv  Detail & Related papers  (2025-06-19T16:18:53Z)
• Guarded Query Routing for Large Language Models [2.2349474636722317]
  We study a guarded query routing problem, for which we first introduce the Guarded Query Routing Benchmark (GQR-Bench)<n>Our results show that WideMLP, enhanced with out-of-domain detection capabilities, yields the best trade-off between accuracy (88%) and speed (4ms)<n>Our findings challenge the automatic reliance on LLMs for (guarded) query routing and provide concrete recommendations for practical applications.
  arXiv  Detail & Related papers  (2025-05-20T15:46:59Z)
• Weighted-Reward Preference Optimization for Implicit Model Fusion [35.57286356489511]
  We propose an implicit fusion method, which leverages preference optimization between the source LLMs and the target LLM to transfer their capabilities effectively.<n>WRPO eliminates the need for vocabulary alignment and matrix fusion and can be efficiently scaled to accommodate various LLMs.<n>Experiments on the MT-Bench, AlpacaEval-2, and Arena-Hard benchmarks demonstrate that WRPO consistently outperforms existing knowledge fusion methods.
  arXiv  Detail & Related papers  (2024-12-04T10:15:12Z)
• MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT [87.4910758026772]
  "Bigger the better" has been the predominant trend in recent Large Language Models (LLMs) development. This paper explores the "less is more" paradigm by addressing the challenge of designing accurate yet efficient Small Language Models (SLMs) for resource constrained devices.
  arXiv  Detail & Related papers  (2024-02-26T18:59:03Z)
• BiLLM: Pushing the Limit of Post-Training Quantization for LLMs [53.31402059062365]
  BiLLM is a groundbreaking 1-bit post-training quantization scheme tailored for pretrained large language models. It achieves for the first time high-accuracy inference (e.g. 8.41 perplexity on LLaMA2-70B) with only 1.08-bit weights across various LLMs families.
  arXiv  Detail & Related papers  (2024-02-06T09:26:34Z)
• BooookScore: A systematic exploration of book-length summarization in the era of LLMs [53.42917858142565]
  We develop an automatic metric, BooookScore, that measures the proportion of sentences in a summary that do not contain any of the identified error types. We find that closed-source LLMs such as GPT-4 and 2 produce summaries with higher BooookScore than those generated by open-source models.
  arXiv  Detail & Related papers  (2023-10-01T20:46:44Z)
• Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting [65.00288634420812]
  Pairwise Ranking Prompting (PRP) is a technique to significantly reduce the burden on Large Language Models (LLMs) Our results are the first in the literature to achieve state-of-the-art ranking performance on standard benchmarks using moderate-sized open-sourced LLMs.
  arXiv  Detail & Related papers  (2023-06-30T11:32:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.