EVENT DOSSIER
RefDiT is a new framework based on reference images, designed to address the issue of difficulty in accurately locating related elements in multi-object scenarios using existing methods. This method uses local region attributes to construct perceptual condition signals by inputting reference images, text prompts, and optional user-guided context. In implementation, RefDiT decomposes identifier tokens at the attribute level to generate condition signals and adjusts the context during inference prompts, which is used to train the LoRA blocks of the generative model based on Diffusion Transformer (DiT). By establishing a correspondence between identifier tokens and local regions of reference images, RefDiT achieves more effective local guidance control.
RefDiT 提出一种基于参考图像生成的新框架，通过局部属性引导解决现有方法无法定位多物体场景相关元素的问题。该方法输入参考图像、文本提示及可选用户指导上下文，利用局部区域属性构建感知条件信号。具体而言，RefDiT 对标识符令牌进行属性级分解以生成条件信号，并在推理提示中进行上下文调整，用于训练基于扩散 Transformer（DiT）的生成模型的 LoRA 块。通过建立标识符令牌与参考图像局部区域的对应关系，RefDiT 实现了更有效的局部引导控制。