Reducing Hallucinated Transcripts in Whisper via Hallucination Space Projection

On September 7, 2026, arXiv cs.AI published a study proposing an unsupervised method to reduce hallucinations in the Whisper model during inference without re-training. This method activates through a low-rank projection decoder, estimates a compact hallucination-related subspace from non-speech calibration data, and projects the decoder’s hidden state outside this subspace during inference. Evaluation showed that the always-on projection scheme reduced the average hallucination rate (HR) of non-speech benchmarks from 31.31% to 2.44%, a relative reduction of 92.21%; while the gating scheme was applied only when the input might be non-speech, reducing HR to 3.74%, a relative reduction of 88.05%, and also resulting in a lower false rejection rate for real speech. On the LibriSpeech dataset, gating projection increased the absolute word error rate (WER) by 0.33-4.39 percentage points, and the false rejection rate (FRR) by 0.41--9.97%. The results indicate that low-rank activation projection can significantly suppress hallucinations in Whisper…