Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original

The researchers proposed a compression model called “Quantization-Aware Healing”. This model operates with only 4-bit precision but still outperforms its full-precision original version. This technology aims to maintain or even improve the inference performance of large language models while significantly reducing computational resource consumption through a quantization-aware repair mechanism. Experiments show that this method successfully addresses the accuracy loss caused by traditional quantization, achieving a balance between efficiency and high performance.