EpicADA: Controllable and Expressive Audio-Driven Animation

EpicADA is a model that generates expressive, realistic animation of the full face, tongue and head from speech audio. Audio features are extracted with a pre-trained Whisper encoder and decoded into animation by recurrent networks, driving MetaHuman rig controls directly. The model is conditioned on emotion, which can either be detected automatically from the voice or set by the user. In a user study, its output was often mistaken for ground-truth performance capture.

The model has shipped in Unreal Engine as MetaHuman audio-driven animation, and continues to be updated.