Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes

Regarding the lossy verification methods in speculative decoding, the study categorizes them into two types: truncated verification and collaborative verification. Through the construction of a cross-benchmark diagnostic evaluation framework, it was found that truncated verification significantly outperforms the true truncated sampling baseline due to distribution distortion; in collaborative verification, overshoot suppression and supervised quality comparison with linear interpolation between the draft and target models are more crucial. This study provides a principled analysis and opens the relevant code for public use.