A Cost-Aware, Paired Protocol for Auditing Dynamic Tool Synthesis in Agentic Video Question Answering

• URL: http://arxiv.org/abs/2607.01469v2
• Date: Fri, 03 Jul 2026 07:03:56 GMT
• Title: A Cost-Aware, Paired Protocol for Auditing Dynamic Tool Synthesis in Agentic Video Question Answering
• Abstract summary: Agentic Video Questioning (VideoQA) systems invoke tools during inference, but their tool libraries are fixed.<n>We propose a cost-aware paired protocol for auditing tool-augmented video agents.
• Score: 2.1096366377985185
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic Video Question Answering (VideoQA) systems invoke tools during inference, but their tool libraries are fixed, so recurring procedures are rebuilt from primitives on every question. Synthesizing composite tools could remove this overhead, but whether such expansion helps is hard to assess: final-answer accuracy, the standard metric, ignores inference effort, so it cannot reveal how a system shifts cost. We propose a cost-aware, paired protocol for auditing tool-augmented video agents. The protocol pairs two complete systems on the same input for each question and reports their net difference across accuracy and cost jointly. For each question, it sorts the paired outcome into one of six groups defined by joint correctness and by the change in visible tool calls, separating accuracy-preserving efficiency gains from harmful regressions. Significance is reported with McNemar's test and paired bootstrap confidence intervals. We instantiate the protocol on Dynamic-SAGE, an agentic VideoQA framework that synthesizes, validates, and persistently registers executable composite tools for reuse on unseen questions, and evaluate it against the SAGE baseline on SAGE-Bench. The audit reveals a multi-axis profile that a scalar accuracy comparison would miss: Dynamic-SAGE improves accuracy by 7.5 points (p < 0.001) and reduces reasoning turns and visible tool calls by roughly 28%, while shifting rather than reducing inference cost, as token usage rises 34% and cost 26%. Gains are largest on visual and open-ended questions and neutral on verbal and multimodal ones, and residual failures concentrate on hard, open-ended questions where the pipeline does the most work. By measuring accuracy and cost jointly, the protocol shows where the pipeline-level difference is reliable and where it is not. The code is available at https://github.com/KurbanIntelligenceLab/Dynamic-SAGE.

Related papers

• AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs [17.842477184182556]
  AsymSpec is an asymmetric speculative decoding framework for agentic pipelines.<n>AsymSpec reaches $approx 90%$ of full-context accuracy on average, delivering $1.3$--$1.7times$ throughput speedups.<n>Results show that asymmetric context access yields substantial gains precisely when compression discards critical reasoning signals.
  arXiv  Detail & Related papers  (2026-08-26T16:50:02Z)
• Disagree to Explore, Agree to Commit: Routing-Guided Test-Time Scaling for Software Agents [60.650808962786364]
  We study whether native MoE router traces can guide steering and selection without an external judge or selection-time test execution.<n>Our analysis shows that routing provides a robust behavioral role signal; token-granular readouts and decision-matched comparison sets turn it into effective control.<n>We evaluate on SWE-bench Verified using open-weight sparse MoE agents across scales and reasoning-effort settings.
  arXiv  Detail & Related papers  (2026-08-23T03:07:03Z)
• Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMs [16.31176289941024]
  We show that Video LLMs improve smoothly or saturate as visual budgets grow.<n>We represent each frozen model--item pair by its response trajectory under controlled visual budgets.<n>We derive matched-grid measures of configuration complementarity, harmful transitions, and text overwrite.
  arXiv  Detail & Related papers  (2026-08-07T09:25:20Z)
• AuditWeave: A Tamper-Evident, Auditor-Navigable Evidence Layer for AI-Assisted and Data-Transformation Workflows [0.0]
  AuditWeave is a lightweight Python library that records the steps of AI-assisted and data-transformation into a single hash-chained ledger.<n>We describe the design and evaluate recording overhead, scalability, and tamper-detection correctness on the reference implementation.
  arXiv  Detail & Related papers  (2026-06-14T21:46:04Z)
• Scaling Laws for Agent Harnesses via Effective Feedback Compute [53.68149869349268]
  emphEffective Feedback Compute (EFC) is a trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained for subsequent decisions.<n>EFC-based coordinates consistently predict failure rates better than raw-compute baselines.
  arXiv  Detail & Related papers  (2026-05-28T09:45:47Z)
• Correction and Corruption: A Two-Rate View of Error Flow in LLM Protocols [51.56484100374058]
  We propose a paired-outcome measurement interface for auditing a single protocol step on exact-match tasks.<n>For each instance, the interface records a baseline correctness bit and a post-step correctness bit.<n>These rates predict accuracy changes and define a reusable empirical interface testable across seeds, mixtures, and pipelines.
  arXiv  Detail & Related papers  (2026-04-20T13:25:40Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Abduct, Act, Predict: Scaffolding Causal Inference for Automated Failure Attribution in Multi-Agent Systems [20.846301581161978]
  Failure attribution in multi-agent systems is a critical yet unsolved challenge.<n>Current methods treat this as a pattern recognition task over long conversation logs.<n>A2P Scaffolding transforms failure attribution from pattern recognition into a structured causal inference task.
  arXiv  Detail & Related papers  (2025-09-12T16:51:15Z)
• How Can Input Reformulation Improve Tool Usage Accuracy in a Complex Dynamic Environment? A Study on $τ$-bench [58.114899897566964]
  In a multi-turn conversational environment, large language models (LLMs) often struggle with consistent reasoning and adherence to domain-specific policies.<n>We propose the Input-Reformulation Multi-Agent (IRMA) framework, which automatically reformulates user queries augmented with relevant domain rules.<n>IRMA significantly outperforms ReAct, Function Calling, and Self-Reflection by 16.1%, 12.7%, and 19.1%, respectively.
  arXiv  Detail & Related papers  (2025-08-28T15:57:33Z)
• AQD: Towards Accurate Fully-Quantized Object Detection [94.06347866374927]
  We propose an Accurate Quantized object Detection solution, termed AQD, to get rid of floating-point computation. Our AQD achieves comparable or even better performance compared with the full-precision counterpart under extremely low-bit schemes.
  arXiv  Detail & Related papers  (2020-07-14T09:07:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.