BOHM: Zero-Cost Hierarchical Attribution for Compound AI Systems

• URL: http://arxiv.org/abs/2605.22866v1
• Date: Tue, 19 May 2026 19:38:14 GMT
• Title: BOHM: Zero-Cost Hierarchical Attribution for Compound AI Systems
• Abstract summary: Compound AI systems route tasks through hierarchies of specialised components.<n>We introduce BOHM, which extracts a hierarchical attribution tree directly from the routing weights such systems already maintain.<n>BOHM satisfies efficiency, monotonicity, symmetry, and weak suppression but not Shapley's additivity.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Compound AI systems route tasks through hierarchies of specialised components. Attribution is dominated by Shapley-based methods (SHAP), which decompose a coalition value function into per-component marginal contributions and require evaluation of the system on arbitrary component subsets. That requirement fails for third-party APIs, opaque endpoints, and agentic orchestrators that concentrate routing on a few tools, leaving most coalitions un-evaluable from the deployed orchestrator. We introduce BOHM, which extracts a hierarchical attribution tree directly from the routing weights such systems already maintain: leaf attribution is the path product of root-to-leaf routing weights; level-k attribution is the induced distribution over depth-k nodes. The method has zero marginal cost, requires no access to component internals, and provides multi-resolution attribution at every level simultaneously, which flat methods cannot offer at any evaluation budget. BOHM and SHAP answer different questions and converge when the deployed router routes near-optimally. On 18 LLMs in a 3-level hierarchy over 880 LiveCodeBench problems, BOHM yields Kendall tau=0.928; SHAP reaches tau=0.980 at 9,000x more coalition evaluations per seed. On a 5-driver, 7-benchmark agentic study (35 cells, complete coverage), drivers concentrate routing on a single tool (top-share median 0.65), and cell-level tau(BOHM,SHAP) is predicted by whether the driver's top pick is the empirically best tool (mean +0.22 vs ~+0.01). On a US Census hierarchy (475 leaves, 4 levels), BOHM recovers ground-truth rankings at every level (tau up to 0.722). BOHM satisfies efficiency, monotonicity, symmetry, and weak suppression but not Shapley's additivity. It is best understood as a complementary primitive: a multi-resolution decomposition computable wherever routing state exists, whose disagreement with Shapley is itself diagnostic.

Related papers

• Search as Computation Allocation [52.080489413778]
  Many algorithms spend an internal resource before returning a decision and are evaluated only by the quality of that terminal output.<n>We formalize such procedures as terminal computation-allocation problems.<n>Bellman equations characterize optimal allocation under fixed budgets, priced computation, and exact certification.
  arXiv  Detail & Related papers  (2026-07-30T08:45:14Z)
• Branching Policy Optimization: Sandbox-Native Language Agent Reinforcement Learning [14.640786318359709]
  Branching Policy Optimization (BPO) is a sandbox-native RL algorithm that snapshots the sandbox at high-entropy decision points along a backbone trajectory.<n>We prove this estimator is unbiased and has strictly lower variance than the trajectory-level baseline.<n>BPO improves success by 3.6--6.1 absolute points over GRPO and RLOO at matched compute, halves gradient-norm variance, and matches the best baseline using 38% fewer policy updates.
  arXiv  Detail & Related papers  (2026-07-15T09:37:36Z)
• TriRoute: Unified Learned Routing for Joint Adaptive Attention, Experts, and KV-Cache Allocation [0.0]
  TriRoute is a single lightweight controller shared across all three axes.<n>It trains end-to-end via a heterogeneous relaxation.<n>It allocates full attention and high-precision cache to sentence-initial positions, rare subwords, and named entities.
  arXiv  Detail & Related papers  (2026-07-07T00:12:46Z)
• Orchard: An Open-Source Agentic Modeling Framework [124.68499958175111]
  We present Orchard, an open-source framework for scalable agentic modeling.<n>At its core is Orchard Env, a lightweight environment service providing reusable primitives for sandbox lifecycle management.<n>On top of Orchard Env, we build three agentic modeling recipes.
  arXiv  Detail & Related papers  (2026-05-14T16:35:12Z)
• ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox [61.862814740220806]
  $textbfComplexMCP$ is a benchmark designed to evaluate agents in rigorous conditions.<n>Built on the Model Context Protocol (MCP), $textbfComplexMCP$ provides over 300 meticulously tested tools.
  arXiv  Detail & Related papers  (2026-05-11T16:20:51Z)
• More Is Not Always Better: Cross-Component Interference in LLM Agent Scaffolding [4.738949927143789]
  LLM agent systems are built by stacking components (planning, tools, memory, self-reflection, retrieval)<n>We run a full factorial experiment over all 25=32 subsets of five components on HotpotQA and GSM8K with Llama-3.1-8B/70B conditions.<n>The All-In system is consistently suboptimal: on HotpotQA, a single-tool agent surpasses All-In by 32% (F1 0.233 vs 0.177, p23)<n>Our findings suggest maximally-equipped agent should be replaced by subset selection via interaction-aware analysis.
  arXiv  Detail & Related papers  (2026-05-07T06:01:43Z)
• Metric Unreliability in Multimodal Machine Unlearning: A Systematic Analysis and Principled Unified Score [14.579552536669217]
  Unlearning in Vision-Language Models (VLMs) is required for compliance with the General Data Protection Regulation Code.<n>We present the first study of reliability in multimodal unlearning.<n>We introduce the Unified Quality Score (UQS), a metric with weights derived from each metric's Spearman correlation with the metric distance.
  arXiv  Detail & Related papers  (2026-05-04T04:13:00Z)
• Equifinality in Mixture of Experts: Routing Topology Does Not Determine Language Modeling Quality [0.0]
  We build a sparse Mixture-of-Experts (MoE) architecture using cosine-similarity routing against learned centroids.<n>We find that routing topology does not determine perplexity (PPL)<n>Expert-level and causal controllability -- which coexist with topology-level equifinality -- are explored in a companion paper.
  arXiv  Detail & Related papers  (2026-04-15T21:02:02Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• BlackboxNLP-2025 MIB Shared Task: Exploring Ensemble Strategies for Circuit Localization Methods [64.5040037515574]
  We investigate whether ensembling two or more circuit localization methods can improve performance.<n>In parallel ensembling, we combine attribution scores assigned to each edge by different methods.<n>In the sequential ensemble, we use edge attribution scores obtained via EAP-IG as a warm start for a more expensive but more precise circuit identification method.
  arXiv  Detail & Related papers  (2025-10-08T09:39:40Z)
• CASSPR: Cross Attention Single Scan Place Recognition [43.68230981047338]
  Place recognition based on point clouds (LiDAR) is an important component for autonomous robots or self-driving vehicles. Current SOTA performance is achieved on accumulated LiDAR submaps using either point-based or voxel-based structures. We propose CASSPR as a method to fuse point-based and voxel-based approaches using cross attention transformers.
  arXiv  Detail & Related papers  (2022-11-22T19:18:30Z)
• Coded Stochastic ADMM for Decentralized Consensus Optimization with Edge Computing [113.52575069030192]
  Big data, including applications with high security requirements, are often collected and stored on multiple heterogeneous devices, such as mobile devices, drones and vehicles. Due to the limitations of communication costs and security requirements, it is of paramount importance to extract information in a decentralized manner instead of aggregating data to a fusion center. We consider the problem of learning model parameters in a multi-agent system with data locally processed via distributed edge nodes. A class of mini-batch alternating direction method of multipliers (ADMM) algorithms is explored to develop the distributed learning model.
  arXiv  Detail & Related papers  (2020-10-02T10:41:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.