DnA: Denoising Attention for Visual Tasks

On September 7, 2026, arXiv published a paper titled “Denoising Attention for Visual Tasks” (arXiv:2606.27372v2), introducing a denoising attention mechanism called DnA. This method aims to address the noise issue caused by the softmax function in traditional multi-head attention mechanisms. Its core principle involves using positive and negative queries to project image features into two subspaces with larger principal angles, thereby enhancing the discriminative ability of the features. Experiments were conducted using the ViT-B backbone network, and results showed that DnA improved performance by 0.8% compared to baseline models on the ImageNet-1K benchmark; in video understanding tasks, it increased the performance of video Transformers by 1.8% and video LLMs by 0.5%.