BOHM Framework Introduces Zero-Cost Hierarchical Attribution for Evaluating Compound AI Systems and Agents

The BOHM framework addresses a critical limitation in current AI evaluation techniques where systems rely on specialized components and third-party APIs. Traditional Shapley-based methods require evaluating numerous component subsets which is often impossible for opaque endpoints or agentic orchestrators. BOHM bypasses this by utilizing existing routing weights to calculate leaf and level-specific attribution directly from the system architecture. This approach enables multi-resolution attribution at every level of a hierarchy simultaneously without additional evaluation cycles or internal access to components. Research conducted across 18 LLMs and 880 programming problems demonstrated that BOHM achieves high correlation with SHAP results while operating at a fraction of the computational budget. These findings suggest that developers can monitor and attribute system performance in real-time by leveraging the inherent routing logic within their AI agents. The study highlights that BOHM and SHAP converge when routers operate near-optimally, providing a viable path for scalable performance monitoring in complex production environments.