A Sim-to-Real Study of Surface-Code Decoder Benchmarking

The researchers conducted benchmark tests based on real data using the Willow processor to evaluate six surface code decoders. The study constructed a ladder consisting of four noise models, covering three code distances, two base groups, and fifteen rounds of counting. The results showed that when each operation type was assigned an independent error rate by the noise model, the rankings aligned with hardware results; while the calibration model reduced the absolute error rate, it did not improve the consistency of rankings. Additionally, NVIDIA’s Ising pre-decoder was independently evaluated for its performance outside the training domain. In 278 out of 280 evaluations, other decoders in the panel achieved or outperformed its performance in terms of per-cycle error rate and decoding latency. The research team has publicly shared the complete evaluation pipeline and specific results of each test to facilitate future comparisons between different decoders and devices.