Language Models Can Control Their Own Attention [R]
2026-09-05 14:07Models🔥 42.2 heat score
1sources
1days unfolding
42.2heat score
2mentions
SummaryAI generated
The research team proposed the Declarative Attention (DA) protocol, which uses chain-of-thinking to guide language models to identify areas of attention. This protocol divides generation into three modes: global, focused, and local, allowing the inference engine to skip most KV cache reads. In zero-sample evaluations on 15 long context tasks based on Gemma-4-31B and Qwen-3.6-27B, DA significantly reduced the total number of attention tokens during decoding (52.0% and 31.1%, respectively), but accompanied by a slight and decreasing accuracy decline with increasing model size (1.27pp and 2.75pp, respectively).
In long context dialogues, language models focus only on a small number of key tokens, yet they still need to scan the entire KV cache. The research team proposed the Declarative Attention (DA) protocol, which uses chain-of-thinking to guide the model in identifying attention regions. This divides generation into three modes: global, focused, and local, allowing the inference engine to skip most KV cache reads. In zero-sample evaluations on 15 long context tasks, DA based on Gemma-4-31B and Qwen-3.6-27B significantly reduced the number of tokens focused during the decoding process (52.0% and 31.1%, respectively), accompanied by a slight and decreasing accuracy drop as the model size increased (1.27pp and 2.75pp, respectively).