PerfReasoning: How Well Do LLMs Reason on Hardware Performance?

On September 7, 2026, arXiv released the PerfReasoning benchmark, aimed at evaluating the capabilities of large language models in hardware performance reasoning and code generation. The benchmark requires models to be compared and predicted based on workload, architecture, and mapping specifications to analyze off-chip traffic and buffer requirements. Test results showed that the most powerful closed-source model scored over 90% in reasoning问答 tasks, while the best open-source model achieved 82.4%; however, performance varied greatly in model construction. The GPT-5.6 Sol model had a pass rate of over 80%, while the average for other configurations was less than 15%. Reinforcement learning for specific tasks improved the mapping reasoning accuracy of the 4B model by 15.7 points, while multi-round self-revision prompts without feedback were unreliable. This benchmark revealed the gap between reasonable architecture reasoning and reliable performance modeling, and will publish the benchmark to support reproducible evaluation and track future progress.