GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Serving

Long-output reasoning makes KV cache a bottleneck for efficient LLM inference services. Existing KV compression methods typically rely on preset request-level budgets, determining only which KV states to retain, while the total KV capacity remains unchanged throughout the decoding process. However, inference workloads exhibit significant changes in request-level demands: different requests require different KV capacities, and the attention demands of individual requests evolve during the generation process. We propose GrowPage, a demand-based KV budget framework that treats KV capacity as runtime resources. GrowPage maintains lightweight dual-time-scale query summaries to capture short-term and long-term attention behaviors and use these summaries to estimate demand evolution. At each capacity boundary, GrowPage dynamically adjusts KV allocation strategies, thereby significantly reducing KV cache overhead while maintaining inference performance.