Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue

On September 7, 2026, the Motion-Omni model was simultaneously released by arXiv and Hugging Face Papers. This research proposes a end-to-end framework for jointly generating speech and full-body movements (including facial, hand, and upper and lower body parts), aiming to address the issue of lack of multimodal synergy in traditional cascading approaches. The model uses Qwen2.5-7B-Instruct as its backbone network, directly extracting motion features from the hidden state of speech, and forces joint training of the LLM, speech generator, and action generator by freezing the speech path. Its supervised data comes from a scalable pipeline containing 422,856 pairs of high-quality pairs (with a total of 1,402 hours). In public evaluations, the Motion-Omni-Q7 instance achieved a gap of less than 2% in reference-independent motion metrics compared to teacher-cascaded systems, with a 5.4-fold increase in inference speed (RTF=0.78), and surpassed all non-teacher-cascaded systems in beat relevance and diversity, with a word error rate of only 2.62%.…