What Moves? Localized Motion Representations for Compositional Scene Control

Researchers published a research paper on arXiv regarding a method for representing local motion. This method aims to address the issue that existing video representations only encode global motion and lack representation of individual entities’ local motion. Its core mechanism involves defining user-specified regions through spatial masks, and directly applying conditional motion encoding to the entire video, rather than using cropped inputs or post-processing mask features. The embeddings generated by this method possess temporal consistency and regional targeting. Experimental results show that this technique performs better than global representation methods based on cropping or post-processing masks in tasks such as object-level motion transfer and local action classification in multi-actor videos, effectively improving controllability while retaining global context information used to eliminate ambiguity.