At the IJCAI 2024 conference, OPPO AI Center's latest research paper, "Zero-shot High-fidelity and Pose-controllable Character Animation," was prominently featured, once again showcasing OPPO's extensive tech expertise and continuous advancement in the AI field.
Currently, in the I2V (Image-to-Video) domain, the most common issues are inconsistent character appearances and poor detail retention, requiring vast amounts of video data for training and consuming significant computational resources.
To address these challenges, OPPO AI Center introduced PoseAnimate, an innovative zero-shot I2V framework. PoseAnimate leverages pose information to control the animation's movements while maintaining scene content consistency.
This framework achieves its goals through three key modules:
● Pose-Aware Control Module (PACM): Integrates various pose signals into text embeddings to retain content irrelevant to the character and ensure precise alignment of movements.
● Dual Consistency Attention Module (DCAM): Enhances temporal consistency, preserving character identity and intricate background details.
● Mask-Guided Decoupling Module (MGDM): Improves the fidelity of generated videos by decoupling characters from the background.The OPPO team also proposed the Pose Alignment Transition Algorithm (PATA) to ensure alignment and smooth transitions of character poposes
Experimental results demonstrate that PoseAnimate significantly outperforms existing advanced training methods in character consistency and detail fidelity, maintaining high temporal consistency throughout the generated animations.
Additionally, Dr. Yang Zhenyu from OPPO AI Center was invited to give a presentation at the IJCAI 2024 conference on "Generative AI for smartphone:
Challenges, Opportunities and Practices of OPPO." He shared OPPO's practical experiences with AI industry partners and academia, contributing to the global exploration of how the next generation of smartphones can responsibly and efficiently integrate AI technology.
