FunnelAudit: Responsibility Auditing in Multi-Route Recommender Systems

• URL: http://arxiv.org/abs/2609.06964v1
• Date: Mon, 07 Sep 2026 03:01:07 GMT
• Title: FunnelAudit: Responsibility Auditing in Multi-Route Recommender Systems
• Abstract summary: Multi-route recommender systems combine retrieval, allocation, fusion, and ranking, making individual inclusions and exclusions difficult to audit.<n>We introduce FunnelAudit, an executable framework for incident-level responsibility auditing.<n>We instantiate the framework in two-stage, nine-route funnels using fixed union, weighted quota allocation, or weighted reciprocal-rank fusion, followed by SASRec ranking.
• Score: 6.810073531792462
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-route recommender systems combine retrieval, allocation, fusion, and ranking, making individual inclusions and exclusions difficult to audit. Route overlap can hide effects from one-at-a-time ablations, while freezing downstream stages produces counterfactuals inconsistent with serving behavior. We introduce FunnelAudit, an executable framework for incident-level responsibility auditing. An accountability contract specifies the disputed Top-K event, controls and owners, permitted reference actions, and replay semantics. FunnelAudit evaluates every permitted control configuration and applies graded actual responsibility to find the smallest outcome-preserving contingency that makes each control pivotal. Its certificate records the contingency and paired serving executions needed to verify the judgment. We instantiate the framework in two-stage, nine-route funnels using fixed union, weighted quota allocation, or weighted reciprocal-rank fusion, followed by SASRec ranking. Across 258,809 user-target incidents from three real interaction datasets, 4.24-16.24% admit a responsible control. Among responsible incident-control pairs, 92.55-99.64% require a nonempty contingency, so single-control ablation recovers only 0.36-7.45%. Policies differing in factual outcomes on only 0.31-2.39% of incidents yield 21.44-54.05% Jaccard distance between responsible-route sets on matched exclusions. Independent replay reproduces all 9,121,792 checked target-world outcomes; exhaustive search and a generic mixed-integer linear program agree with every sampled judgment. These findings demonstrate the importance of explicit serving semantics and checkable witnesses for recommender accountability.

Related papers

• Guardrailed Meta-Agent Loops: Stress-Testing Policy Pinning, Budget Bounds, and Crash Recovery [7.414694666903067]
  GuardrailLoop is a simulation-based testbed that makes three operational contracts jointly testable.<n>A hash-pinned policy fixes goals, scope, evaluation identity, budget, and release conditions.<n>It separates useful adaptation, state recovery, and repeated execution.
  arXiv  Detail & Related papers  (2026-09-10T21:22:08Z)
• Ranked by the Matcher: A Reproducibility Audit of Knowledge Graph Extraction from Threat Reports [5.580052437190499]
  Re-scoring ten system outputs on shared documents under eight protocols reverses eleven of forty-five pairwise orderings.<n>On GRID's external 378-item calibration set, no mechanical matcher agrees with multi-reviewer adjudication above 71%.<n> validation raises precision for all four hosted backbones and lowers it for all three offline backbones.
  arXiv  Detail & Related papers  (2026-09-01T04:45:44Z)
• RecurSE: Bounded Recursive Self-Evaluation for LLM Rubric Judges [48.831446371402414]
  RecurSE pairs a trainable judge with a synchronized policy-copy checker that audits the judge's reasoning against meta-rubrics to supply a scalar process reward.<n>We show that RecurSE achieves consistent generalization gains across held-out medical, pairwise, summarization, and professional benchmarks.
  arXiv  Detail & Related papers  (2026-08-25T08:35:04Z)
• Disagree to Explore, Agree to Commit: Routing-Guided Test-Time Scaling for Software Agents [60.650808962786364]
  We study whether native MoE router traces can guide steering and selection without an external judge or selection-time test execution.<n>Our analysis shows that routing provides a robust behavioral role signal; token-granular readouts and decision-matched comparison sets turn it into effective control.<n>We evaluate on SWE-bench Verified using open-weight sparse MoE agents across scales and reasoning-effort settings.
  arXiv  Detail & Related papers  (2026-08-23T03:07:03Z)
• Cross-View Correspondence Is a Measurement Intervention: Two-Sided Validation for Agent Evaluation and Credit Assignment [3.2479665905442645]
  Cross-view correspondence must be declared, validated, and propagated into uncertainty before agent evaluation or credit assignment supports a point conclusion.<n>We develop a validity theory and audit with three components: two-sided validation of nuisance removal and response preservation, all-optima identification of downstream conclusions, and uncertainty propagation after validity is established.
  arXiv  Detail & Related papers  (2026-08-18T12:39:21Z)
• When Shared Rollouts Fail in Defensive Driving Evaluation: A NAVSIM Score Basis Audit [2.823425836233287]
  Re-simulation benchmarks may use reference-conditioned forgiveness, under which an agent receives credit when a logged human reference fails a compliance channel.<n>We audit this risk in NAVSIM v2.2 original scene single-stage scoring.
  arXiv  Detail & Related papers  (2026-08-05T14:22:23Z)
• SEER: A Self-Grounded Evidence Interface for Controlled Spatial Relation Classification [71.9783246097687]
  SEER hides candidate relations during pair localization, constructs a query-specific view with explicit subject/object roles, and retains the full image and sparse box geometry as complementary evidence.<n>For relation-choice protocols with exact inverse support, an optional refinement swaps the entity roles and changes the forward decision only when exactly one visual state obeys the corresponding inverse relation.<n>The unchanged protocol yields +4.35 to +11.79 on all 2,434 filtered EmbSpatial pair-relation questions across three models.
  arXiv  Detail & Related papers  (2026-08-04T13:16:15Z)
• Partial Evidence Bench: Benchmarking Authorization-Limited Evidence in Agentic Systems [0.0]
  This paper introduces Partial Evidence Bench, a deterministic benchmark for measuring that failure mode.<n>The benchmark ships three scenario families -- due diligence, compliance audit, and security incident response.<n>It evaluates systems along four surfaces: answer correctness, completeness awareness, gap-report quality, and unsafe completeness behavior.
  arXiv  Detail & Related papers  (2026-05-06T19:01:29Z)
• SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation [6.604874054866016]
  We present SURE-RAG, a transparent aggregation protocol built on the observation that evidence sufficiency is a set-level property.<n>A shared pair-level claim-evidence verifier produces local relation distributions, which SURE-RAG aggregates into interpretable answer-level signals.<n>We evaluate on HotpotQA-RAG v3, a controlled multi-hop benchmark, under an artifact-aware protocol.
  arXiv  Detail & Related papers  (2026-05-05T09:05:40Z)
• IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction [72.51952455865155]
  We present IMPACT-HOI, a mixed-initiative framework for annotating egocentric procedural video.<n> IMPACT-HOI frames this task as the incremental resolution of a partially specified, onset-anchored event state.<n>A user study with 9 participants shows a 13.5% reduction in manual annotation actions, a 46.67% event match rate, and zero confirmed-field violations.
  arXiv  Detail & Related papers  (2026-05-03T01:37:40Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them [58.04324690859212]
  Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks.<n>We identify two fundamental types of inconsistencies: Score-Comparison Inconsistency and Pairwise Transitivity Inconsistency.<n>We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations.
  arXiv  Detail & Related papers  (2025-09-25T13:04:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.