Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One

The researchers proposed PlaidQ, a continuous diffusion language model based on 0.7B parameters, aimed at improving code generation efficiency through distillation techniques. This model transforms pre-trained autoregressive models into bidirectional denoising filters with continuous word embeddings, and is trained using distribution matching and pairwise trajectory supervision. Experiments show that PlaidQ is competitive with discrete diffusion models at the same scale; its 16-step student model achieved pass@10 scores of 31.78 and 40.49 on the HumanEval and MBPP+ test sets, respectively, outperforming the teacher model with 512 steps of sampling. Additionally, extreme single-step denoising distillation enabled the model to generate functionally correct programs on the HumanEval with a pass@1 score of 7.07. These results confirm the effectiveness of continuous diffusion models as a viable approach for code generation with fewer or even single steps.