Motion-aware fine-tuning of CLIP text encoders for zero-shot human motion generation accuracy

Human motion generation is essential for fields such as animation, robotics, and virtual reality, requiring models that effectively capture motion dynamics from text descriptions. Existing approaches often rely on Contrastive Language-Image Pretraining (CLIP)-based text encoders, but their training on text-image pairs constrains their ability to understand temporal and kinematic structures inherent in motion and motion generation. This work introduces MoCLIP, a fine-tuned CLIP model with an additional motion encoding head, trained on motion sequences using contrastive learning and tethering lo

Research goal: To what extent does motion-aware fine-tuning of CLIP text encoders improve zero-shot accuracy on human motion generation benchmarks compared to standard image-pretrained CLIP?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 8.7/10.