EVENT DOSSIER
To address the memory bottleneck issue caused by the linear increase in KV cache due to the increasing sequence length of large reasoning models (LRMs), the research team proposed the BeaconKV method. This method requires no training and predicts key-value pairs that will be revisited by maintaining “beacons” queries representing the global query cluster. By utilizing the characteristic of思维的 revisit markers in long-range reasoning, which cluster into a small number of similar groups in the embedding space, BeaconKV replaces recent queries to estimate the importance of future tokens. Experimental results show that BeaconKV performs well across four open-source LRMs and various reasoning benchmarks, achieving up to 5.8 times more memory reduction compared to existing compression methods, while almost maintaining full-cache accuracy and increasing throughput by more than 4.3 times.
BeaconKV 提出基于信标查询的 KV 缓存压缩方法
BeaconKV 提出一种无需训练的 KV 缓存压缩方法，通过维护代表全局查询簇的“信标”查询来减少推理大模型内存占用。该方法利用长程推理中 Thought Revisiting Tokens（TRT）在嵌入空间中聚类为少量相似组的特性，…
Large Reasoning Models (LRMs) 因 KV 缓存随序列长度线性增长而面临严重的内存瓶颈。BeaconKV 提出一种无需训练的关键值缓存压缩方法，通过维护代表全局查询簇的“信标”查询来预测将被重访的 KV 对，避免…
BeaconKV 提出一种无需训练的 KV 缓存压缩方法，通过维护代表全局查询簇的“信标”查询来减少推理大模型内存占用。该方法利用长程推理中 Thought Revisiting Tokens（TRT）在嵌入空间中聚类为少量相似组的特性，仅存储关键代表而非完整历史。实验表明，BeaconKV 在四个开源推理大模型及多种基准测试中表现优异，相比现有方法最高减少 $5.8\times$ 内存占用，同时保持接近全缓存的准确率并将吞吐量提升超过 $4.3\times$。
Large Reasoning Models (LRMs) 因 KV 缓存随序列长度线性增长而面临严重的内存瓶颈。BeaconKV 提出一种无需训练的关键值缓存压缩方法，通过维护代表全局查询簇的“信标”查询来预测将被重访的 KV 对，避免存储完整查询历史。该方法基于长程推理中 Thought Revisiting Tokens (TRT) 在嵌入空间中聚类为少量相似组的发现，利用这些信标查询替代近期查询以估计未来 token 重要性。实验表明，BeaconKV 在四个开源 LRMs 及多样化推理基准测试中表现优异，相比现有压缩方法实现了高达 $5.8\times$ 的内存缩减，同时几乎保持全缓存精度并将吞吐量提升超过 $4.3\times$。