$A^2E$ : An End-to-End Agent Auditing Engine

On September 7, 2026, researchers published $A^2E$ (Agent Auditing Engine) on arXiv. This is an end-to-end evaluation engine designed specifically for agent testing frameworks. The engine utilizes the newly proposed Agent Task Protocol (ATP) to quickly integrate evaluation tasks with various frameworks and captures standardized execution trajectories through automatic instrument monitoring. During the evaluation phase, $A^2E$ systematically assesses framework capabilities using a multi-dimensional set of metrics, allowing for a more detailed analysis of differences in execution efficiency, tool usage, task planning, and error recovery compared to single correctness metrics. Experiments show that combinations of models and frameworks exhibit significant performance fluctuations across different types of tasks, and no single combination consistently performs best across all tasks. These findings demonstrate the necessity of systematic evaluation and provide guidance for the co-evolution of models and frameworks. The relevant code has been published on GitHub.