WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systems

• URL: http://arxiv.org/abs/2605.07068v1
• Date: Fri, 08 May 2026 00:25:16 GMT
• Title: WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systems
• Abstract summary: We characterize the compilation gap across 17 RepLiQA domains (6,800 questions)<n>We propose WiCER, an iterative algorithm inspired by counterexample-guided abstraction refinement (CEGAR) that closes this gap.<n>An ablation across all 17 topics confirms that targeted diagnosis (+0.95), not generic pinning (+0.16), drives the gains.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The LLM Wiki pattern, to compile and provide domain knowledge into a persistent artifact and serve it to LLMs via KV cache inference, promises context access at sub-second latency with zero retrieval failure. Realizing this requires solving the compilation gap: LLM compilation distilling raw documents into a wiki without catastrophically discarding critical facts. We characterize this gap across 17 RepLiQA domains (6,800 questions): we observe that full context KV cache inference outperforms RAG on curated knowledge (4.38 vs. 4.08 out of 5, 7.3 faster TTFT) but degrades below RAG at scale due to attention dilution, and blind compilation fails entirely (2.14 to 2.32 vs. 3.46, 53 to 60% catastrophic failure rate). To address the compilation gap, we propose WiCER (Wiki-memory Compile, Evaluate, Refine), an iterative algorithm inspired by counterexample-guided abstraction refinement (CEGAR) that closes this gap. WiCER evaluates compiled wikis against diagnostic probes, identifies dropped facts, and forces their preservation in subsequent compilations. One to two iterations recover 80% of lost quality (mean 3.24 vs. 3.47 for raw full-context across the 15 topics with baselines), reducing catastrophic failures by 55% relative. An ablation across all 17 topics confirms that targeted diagnosis (+0.95), not generic pinning (+0.16), drives the gains. All code and benchmarks are released for reproducible research.

Related papers

• IdeaAMBIG: Benchmarking Implementation-Critical Gaps in Research-Idea Specifications [52.570108663867046]
  A research idea may be novel, coherent, and scientifically plausible, yet its proposed method may remain insufficiently specified for faithful implementation.<n>We study the codification readiness of implementation-facing research-method specifications, defined by whether they provide sufficient methodological information for a competent implementer or coding agent to construct the intended method without unsupported assumptions.<n>We introduce IdeaAMBIG, a benchmark of 660 evidence-grounded instances: 163 real-world gaps from reports and GitHub issues, and 497 controlled synthetic gaps injected into codification-ready references.
  arXiv  Detail & Related papers  (2026-09-09T17:59:04Z)
• From Runnable to Verifiable: An Independent Reproducibility Study of LLM/Agent-Driven Vulnerability Validation Artifacts [4.093009863847175]
  Security research artifacts are increasingly produced by agent-driven vulnerabilities.<n>The gap between emphpublicly available, emphrunnable, emphsignal-producing, and emphsemantically confirmed artifacts is poorly measured.<n>We conduct a pre-registered consensus audit of this literature.
  arXiv  Detail & Related papers  (2026-08-10T13:05:02Z)
• Control-Plane Placement Shapes Forgetting: An Architectural Study of Agent Memory Across Thirteen System Configurations [1.2763567932588586]
  ForgetEval is a 1000-case templated suite plus a 385-case adversarial layer (132 hand-crafted + 253 LLM-drafted oracle-validated)<n>We observe three placement regimes with partly complementary coverage: deterministic primitives suffice for lexical/temporal categories but fail canonicalization.<n>We expose the trade-off via ForgetEval, a 1000-case templated suite plus a 385-case adversarial layer.
  arXiv  Detail & Related papers  (2026-06-14T16:32:15Z)
• How Far Is Document Parsing from Solved? PureDocBench: A Source-TraceableBenchmark across Clean, Degraded, and Real-World Settings [56.70440596502351]
  The past year has seen over 20 open document parsing models, yet benchmarks almost exclusively on OmniDocBench.<n>We present PureDocBench, a benchmark thatrenders document images from HTML/CSS, covering 10 domains, 66 pages, and 1,475 pages.
  arXiv  Detail & Related papers  (2026-05-08T09:30:31Z)
• Constraint-Guided Multi-Agent Decompilation for Executable Binary Recovery [47.704311990064554]
  Decompilation is essential for security analysis, malware reverse engineering, and legacy software maintenance.<n>We present a multi-agent framework that transforms decompiled code into re-executable source through Multi-level Constraint-Guided Decompilation (MCGD)<n>Our framework achieves 84-97% re-executability, improving baseline decompiler output by 28-89 percentage points.
  arXiv  Detail & Related papers  (2026-04-27T01:28:11Z)
• Self-Aware Vector Embeddings for Retrieval-Augmented Generation: A Neuroscience-Inspired Framework for Temporal, Confidence-Weighted, and Relational Knowledge [0.0]
  Modern retrieval-augmented generation (RAG) systems treat vector embeddings as static, context-free artifacts.<n>We propose Smart, a framework that augments dense embeddings with three explicit properties -- temporal awareness, confidence decay, and relational awareness.
  arXiv  Detail & Related papers  (2026-04-22T14:13:50Z)
• Mitigating LLM Hallucinations through Domain-Grounded Tiered Retrieval [0.0]
  Large Language Models (LLMs) have achieved unprecedented fluency but remain susceptible to "hallucinations"<n>We propose a tiered retrieval and verification architecture designed to intercept factual inaccuracies by shifting LLMs from pattern-matchers to verified truth-seekers.<n>System was evaluated across 650 queries from five diverse benchmarks.
  arXiv  Detail & Related papers  (2026-03-18T15:59:30Z)
• Selective Memory for Artificial Intelligence: Write-Time Gating with Hierarchical Archiving [0.0]
  Parametric approaches compress knowledge into weights, precluding selective updates.<n>We introduce write-time gating that filters incoming knowledge objects using composite salience scores.
  arXiv  Detail & Related papers  (2026-03-16T23:06:44Z)
• Resource-Efficient Iterative LLM-Based NAS with Feedback Memory [49.44875022114861]
  Neural Architecture Search (NAS) automates network design, but conventional methods demand substantial computational resources.<n>We propose a closed-loop pipeline leveraging large language models (LLMs) to iteratively generate, evaluate, and refine convolutional neural network architectures.
  arXiv  Detail & Related papers  (2026-03-12T16:00:22Z)
• Detecting Corpus-Level Knowledge Inconsistencies in Wikipedia with Large Language Models [11.16952630564181]
  We focus on inconsistencies, a specific type of factual inaccuracy, and introduce the task of corpus-level inconsistency detection.<n>We present CLAIRE, an agentic system that combines LLM reasoning with retrieval to surface potentially inconsistent claims along with contextual evidence for human review.<n>In a user study with experienced Wikipedia editors, 87.5% reported higher confidence when using CLAIRE, and participants identified 64.7% more inconsistencies in the same amount of time.
  arXiv  Detail & Related papers  (2025-09-27T10:32:41Z)
• Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory [52.44029486173232]
  Dynamic Cheatsheet (DC) is a lightweight framework that endows a black-box language model with a persistent, evolving memory.<n>DC enables models to store and reuse accumulated strategies, code snippets, and general problem-solving insights at inference time.<n>This test-time learning enhances performance substantially across a range of tasks without needing explicit ground-truth labels or human feedback.
  arXiv  Detail & Related papers  (2025-04-10T17:57:33Z)
• LLM Robustness Against Misinformation in Biomedical Question Answering [50.98256373698759]
  The retrieval-augmented generation (RAG) approach is used to reduce the confabulation of large language models (LLMs) for question answering. We evaluate the effectiveness and robustness of four LLMs against misinformation in answering biomedical questions.
  arXiv  Detail & Related papers  (2024-10-27T16:23:26Z)
• WikiContradict: A Benchmark for Evaluating LLMs on Real-World Knowledge Conflicts from Wikipedia [59.96425443250666]
  Retrieval-augmented generation (RAG) has emerged as a promising solution to mitigate the limitations of large language models (LLMs) In this work, we conduct a comprehensive evaluation of LLM-generated answers to questions based on contradictory passages from Wikipedia. We benchmark a diverse range of both closed and open-source LLMs under different QA scenarios, including RAG with a single passage, and RAG with 2 contradictory passages.
  arXiv  Detail & Related papers  (2024-06-19T20:13:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.