What Drives Recovery in Agentic Text-to-Cypher? LAST-CQ: An LLM Agent Self-Refinement Framework

• URL: http://arxiv.org/abs/2609.12746v1
• Date: Fri, 11 Sep 2026 11:47:42 GMT
• Title: What Drives Recovery in Agentic Text-to-Cypher? LAST-CQ: An LLM Agent Self-Refinement Framework
• Abstract summary: LAST-CQ is a five-agent, training-free, execution-grounded Text-to-Cypher framework.<n>We run three counterfactuals over 2,471 live-Database queries and six backbones spanning three vendor scale tiers.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic pipelines for structured-query generation are rapidly expanding, but it is unclear which part of the loop produces the gain. We use LAST-CQ -- a five-agent, training-free, execution-grounded Text-to-Cypher framework -- as an instrumented testbed, running three counterfactuals over 2,471 live-database queries and six backbones spanning three vendor scale tiers. Removing correction is worth between 3.1% aggregate execution-BLEU against the single-pass system and 12.3% against a no-refinement counterfactual (up to 80.7% for the weakest backbone). Replacing schema-grounded, LLM-synthesised feedback with raw database error strings costs almost nothing (20.9% vs. 19.9% naive exact match; <0.2% end-to-end; equivalent within $\pm 0.075$ set-F1 by two one-sided tests). Spending the same call budget on parallel sampling degrades quality by 10-11%. What works is detecting failure and routing it to a retry, not the feedback sophistication or number of samples. LAST-CQ itself recovers 91.7% of queries that fail under single-pass generation, while a query that succeeds first time still costs exactly one LLM call. We also show that n-gram overlap on serialised results is not a bound in either direction: it over-scores against set equivalence on 65.9% of results while under-scoring against judged semantics. Finally, we calibrate our LLM judge against blind human labels and find it optimistic by 9 points.

Related papers

• Your Retriever Already Knows: Distribution-Shape QPP for RAG Retrieval Sufficiency [0.0]
  Motivated by a Czech nuclear-regulator deployment, we compare three Query Performance Prediction (QPP) paradigms for retrieval sufficiency in RAG.<n>Our 24 non-lexical features reach a weighted-average AUROC of 0.856 at 2 ms per query, ahead of a classic-QPP literature pool.<n>Adding the LLM judgment as one feature (hybrid) matches S1 on ViDoRe (0.863) but gains a statistically significant edge on SJB.
  arXiv  Detail & Related papers  (2026-09-10T14:52:16Z)
• Codebook Agent: Amortized Topology Design for LLM Multi-Agent Systems [67.04448659688579]
  We develop a query-independent 16-entry codebook that ranks the top decoded candidates in a single batched forward pass.<n>With no iterative search and no message passing at test time, Codebook Agent is the most accurate method on all six benchmarks.
  arXiv  Detail & Related papers  (2026-09-02T08:10:22Z)
• Business Truth, not SQL Accuracy: A Rule-Gated 7B Analytics Agent Outperforms a Direct-Prompted 32B Baseline [0.0]
  This paper introduces WarehouseBench, 400 frozen tasks over two synthetic warehouses in which roughly half the correct responses are a clarification, an abstention or a refusal.<n>QueryProof, a 7B agent, uses rules derived from a semantic layer and physical catalog to determine its behaviour.
  arXiv  Detail & Related papers  (2026-08-10T08:14:24Z)
• A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination [51.06539604709775]
  We propose A-SR, a self-evolving agentic framework that shifts the control unit from expression edits to role-conditioned evidence views.<n>A-SR coordinates formula discovery through routing among coordination protocols, an online evaluator-reward role policy, and state-routed process memory.
  arXiv  Detail & Related papers  (2026-08-05T14:01:10Z)
• Real-Time Detection and Repair of LLM Agent Failures [0.0]
  LLM agents fail mid-episode loop, cascade tool errors, drift off goal, fabricate results, or silently absorb corrupted content.<n>We ask how much detection is achievable from observable step alone, using monitors costing microseconds per step and trained only on healthy runs.<n>System runs at 200 microseconds per step, three orders of magnitude below a judge call.
  arXiv  Detail & Related papers  (2026-08-03T16:34:46Z)
• StaminaBench: Stress-Testing Coding Agents over 100 Interaction Turns [45.43677974796221]
  We introduce StaminaBench, a benchmark that measures the stamina of coding agents.<n>Unlike the prevailing fraction-tasks metric, this matches real vibecoding where sessions run dozens or hundreds of turns.
  arXiv  Detail & Related papers  (2026-06-17T21:36:09Z)
• Layer-Isolated Evaluation: Gating the Deterministic Scaffold of a Production LLM Agent with a No-LLM, Regression-Locked Test Harness [45.148328075418156]
  A deployed ordering agent is decomposed into a fixed taxonomy of layers.<n>The pure suite runs in CI on every change against a locked per-slice baseline.<n>We validate by controlled regression injection, degrading one layer at a time across seven non-safety layers.
  arXiv  Detail & Related papers  (2026-06-10T05:55:13Z)
• CRANE: Knowledge Editing for Reasoning MLLMs [37.084419383244814]
  multimodal large language models (MLLMs) generate explicit chain-of-thought (CoT) reasoning before producing answers.<n>On reasoning MLLMs, methods that generalize (FT, LoRA) trigger format collapse, while methods without deep modification cannot generalize.<n>We propose CRANE, a retrieval-augmented framework that requires no per-edit parameter modification.
  arXiv  Detail & Related papers  (2026-06-08T05:01:11Z)
• Scaling Laws for Agent Harnesses via Effective Feedback Compute [53.68149869349268]
  emphEffective Feedback Compute (EFC) is a trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained for subsequent decisions.<n>EFC-based coordinates consistently predict failure rates better than raw-compute baselines.
  arXiv  Detail & Related papers  (2026-05-28T09:45:47Z)
• How Code Representation Shapes False-Positive Dynamics in Cross-Language LLM Vulnerability Detection [8.360057179027322]
  How code representation format shapes false positive behaviour in cross-language vulnerability detection remains poorly understood.<n>We systematically vary training intensity and code representation format, comparing raw source text with pruned Abstract Syntax Trees at both training time and inference time.<n>Cross-language FPR reflects the joint effect of training-time and inference-time representation, not either alone.
  arXiv  Detail & Related papers  (2026-04-30T11:01:03Z)
• SQL-ASTRA: Alleviating Sparse Feedback in Agentic SQL via Column-Set Matching and Trajectory Aggregation [51.696027315573296]
  We propose a framework featuring a universal two-tiered reward mechanism designed to provide effective trajectory-level evaluation and dense step-level signals.<n>In traditional paradigms, rewards are determined solely by the final-turn feedback, which ignores the intermediate process and leads to ambiguous credit evaluation.<n>We show that our approach outperforms SOTA Arctic-Text2-R1-7B on BIRD 2.0 using identical models, propelling Text-to-reward convergence toward a robust multi-turn agent paradigm.
  arXiv  Detail & Related papers  (2026-03-17T06:26:36Z)
• VAULT: Vigilant Adversarial Updates via LLM-Driven Retrieval-Augmented Generation for NLI [15.320553375828045]
  VAULT is a fully automated adversarial RAG pipeline that uncovers and remedies weaknesses in NLI models.<n>VAULT consistently outperforms prior in-context adversarial methods by up to 2.0% across datasets.
  arXiv  Detail & Related papers  (2025-08-01T14:22:54Z)
• Reasoning Robustness of LLMs to Adversarial Typographical Errors [49.99118660264703]
  Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning using Chain-of-Thought (CoT) prompting. We study the reasoning robustness of LLMs to typographical errors, which can naturally occur in users' queries. We design an Adversarial Typo Attack ($texttATA$) algorithm that iteratively samples typos for words that are important to the query and selects the edit that is most likely to succeed in attacking.
  arXiv  Detail & Related papers  (2024-11-08T05:54:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.