Learning 3D Editing without Paired Supervision via Generative Prior Distillation

The researchers propose a new framework for feedforward 3D editing that can learn without paired supervision, aiming to address the scarcity of high-quality paired training data. This method does not rely on real 3D paired data; instead, it uses generative prior distillation techniques to transfer visual, semantic, and geometric knowledge from the base model to the 3D editing model. In implementation, it combines a differentiable rendering pipeline, integrating the 2D visual priors of the main editing perspective image model with the semantic priors provided by the novel perspective visual-language model for supervision, ensuring strict adherence to instructions and preservation of source identity features. To address the problems of geometric collapse and multi-perspective inconsistency caused by 2D projection supervision, 3D perceptual distribution matching regularization is introduced as a geometric prior, constraining the editing output to remain within the real 3D asset manifold defined by the pre-trained image to 3D teacher model. Experimental results show that this method performs well in terms of instruction fidelity and cross-perspective consistency, significantly outperforming existing state-of-the-art baselines. The project code is now open-source.