From Answers to Interpretations: Rethinking Ambiguity-Induced Aleatoric Uncertainty Estimation in LLMs

To address the problem of uncertainty estimation in existing large language models (LLMs), which relies on generating multiple clarifications and querying the model for comparative answers, resulting in redundant answers and cognitive leakage, researchers have proposed a new evaluation method. This method does not require answering the clarified inputs; instead, it directly estimates the accidental uncertainty components caused by input ambiguity from the rational interpretation space. In three benchmark tests, this new method improved the AUROC score from 60.85 to 63.34, reduced the cost of calculating output tokens by 4-26 times, decreased the number of API calls by 2.2-3.5 times, and significantly reduced the correlation between estimated values and cognitive uncertainty. The results indicate that estimating accidental uncertainty caused by ambiguity from the interpretation space is more effective than from the response space.