Cache-Aware Joint Router Adaptation for Memory-Efficient MoE Inference

The researchers proposed a cache-aware post-training framework to optimize the inference memory efficiency of MoE models. This framework combines the adaptation of the MoE backbone network with lightweight auxiliary cache routers, improving performance while maintaining the native Top-K expert selection rules. The experiments involved two routing mechanisms: the Temporal Router that only updates information, and the Spatio-Temporal Router that includes spatial routers. The research team evaluated three tasks on Qwen3 and GPT-OSS models: GSM8K, MATH, and CommonsenseQA. The results showed that the Temporal Router consistently improved cache hit rates and reduced expert weight transmission; the Spatio-Temporal Router achieved the best load adjustment efficiency in the three tasks of Qwen3, increasing the adjusted hit rate by 1.15 to 18.03 percentage points compared to the strongest prefetch baseline, and reducing traffic…