EVENT DOSSIER
To address the high failure costs caused by blind actions of LLM agents in software engineering, researchers proposed the Speculative Uncertainty (SU) method. This method utilizes a lightweight open-source draft model that only outputs tokens to perform a single forward scoring on the trajectories generated by the agent, without needing to access logits, weights, or repeated sampling. By separating the inference and action phase features and calibrating them to verifiable targets, SU generates failure probability scores for direct use by downstream strategies. Experiments with Qwen3-Coder-480B and Claude 3.5 Sonnet showed that this method reduced the execution error rate during deployment by 6-8 percentage points and decreased Token costs by 14-19%. Additionally, SU can be migrated to out-of-distribution benchmarks without re-training and achieves cross-model generalization.
LLM 代理在软件工程部署中因盲目行动导致高昂失败成本，研究人员提出 Speculative Uncertainty（SU）方法。该方法利用仅输出 token 的轻量级开源草稿模型，通过单次前向传播对代理已生成轨迹进行评分，无需访问 lo…
研究人员提出 Speculative Uncertainty (SU) 方法，旨在从黑盒智能体输出中恢复预测失败信号。该方法利用小型开源草稿模型对已生成轨迹进行单次前向评分，通过分离推理与行动跨度提取特征并校准为可验证目标，从而生成下游策…
LLM 代理在软件工程部署中因盲目行动导致高昂失败成本，研究人员提出 Speculative Uncertainty（SU）方法。该方法利用仅输出 token 的轻量级开源草稿模型，通过单次前向传播对代理已生成轨迹进行评分，无需访问 logits、权重或重复采样。基于推理与动作阶段的特征分离及可验证目标校准，SU 生成失败概率分数供路由、人工干预等下游策略直接消费。在 Qwen3-Coder-480B 和 Claude 3.5 Sonnet 上的预执行否决门实验显示，该方法将部署时的执行错误率降低 6-8 个百分点，token 成本减少 14-19%，且无需重新训练即可泛化至分布外基准及多种代理模型。
研究人员提出 Speculative Uncertainty (SU) 方法，旨在从黑盒智能体输出中恢复预测失败信号。该方法利用小型开源草稿模型对已生成轨迹进行单次前向评分，通过分离推理与行动跨度提取特征并校准为可验证目标，从而生成下游策略可直接使用的失败概率分数。在软件工程中部署该方法的预执行否决门策略后，Qwen3-Coder-480B 和 Claude 3.5 Sonnet 等智能体的执行错误率降低了 6-8 个百分点，Token 成本减少了 14-19%，且该方法无需重新训练即可迁移至分布外基准并跨模型泛化。