EVENT DOSSIER
September 7, 2026: Sierra Research released the τ^τ-Bench benchmark environment, covering 53 cross-domain tasks…
On September 7, 2026, Sierra Research released the τ^τ-Bench benchmark environment, designed to evaluate the ability of AI agents to build customer service systems in real-world scenarios. The benchmark includes 53 tasks across various domains, requiring the agent to deliver deployable systems based on corporate records, customer needs, production APIs, existing code libraries, and cost constraints. Test results showed that the best-performing configuration (Claude Opus 5 under Claude Code) only passed 23.9% of the evaluations, far below the expert reference score of 82.2%. Experiments identified issues such as shallow querying, lack of communication with customers, and insufficient architectural experimentation. This benchmark aims to turn the construction of collaborative agents into a quantifiable goal.
arXiv:2609.04611v1 $\tau^\tau$-bench 发布端到端真实 Agent 构建基准
September 7, 2026: Sierra Research released the τ^τ-Bench benchmark environment, covering 53 cross-domain tasks…
Sierra Research released τ^τ-Bench, aimed at evaluating the ability of AI agents to build complete customer service agents under real-world customer interaction conditions. The benchmark covers 53 cross-domain tasks, requiring developers to deliver deployable solutions based on business records, customer needs, production APIs, existing code libraries, and cost constraints…
2 reportsSierra Research 发布 τ^τ-Bench，旨在评估编码智能体构建端到端、真实客户场景下客户服务智能体的能力。该基准模拟真实业务环境，提供企业记录、客户需求、生产 API、代码库及成本限制等要素，要求智能体交付完整服务并通过持外模拟用户测试。在涵盖四个领域的 53 项任务中，最强配置 Claude Opus 5 仅通过 23.9% 的评估，而专家参考上限得分为 82.2%。实验发现模型存在查询浅层化、缺乏客户沟通及架构实验不足等问题，旨在将协作智能体构建转化为可量化目标。
$\tau^\tau$-Bench 发布，旨在评估 AI 代理在真实客户互动条件下构建完整客户服务代理的能力。该基准涵盖 53 个跨领域任务，要求开发者基于企业记录、客户需求、生产 API、现有代码库及成本限制交付可部署的代理系统。测试显示，最强配置 Claude Opus 5 under Claude Code 仅通过 23.9% 的评估模拟，而专家参考上限得分为 82.2%。失败模式与人类开发者所见一致：模型进行浅层查询而非深度理解记录、几乎不与客户沟通、且缺乏对代理架构及服务成本的实验，仅交付首个运行设计。