Authors: Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, Bo Dai

Published on: July 10, 2023

Impact Score: 8.22

Arxiv code: Arxiv:2307.04725

Summary

• What is new: Introduces a framework for animating personalized text-to-image models without requiring individual model tuning.
• Why this is important: Adding motion to high-quality personalized text-to-image creations to produce animations is challenging.
• What the research proposes: Proposed AnimateDiff, a plug-and-play motion module, trained with transferable motion priors, that can animate any personalized T2I, and MotionLoRA for fine-tuning new motion patterns.
• Results: Successfully generated smooth animations while maintaining visual quality and motion diversity across various personalized T2I models.

Technical Details

Technological frameworks used: AnimateDiff, MotionLoRA

Models used: Stable Diffusion, DreamBooth

Data used: Real-world videos

Potential Impact

Animation and film production, personalized content creation platforms, advertising agencies

Want to implement this idea in a business?

We have generated a startup concept here: AnimateX.