NVIDIA Omniverse™ Audio2Face is a combination of artificial intelligence (AI)-based technologies that derive facial motion and lip sync animations from an audio source.
With Audio2Face, you can:
Analyze an audio sample and automatically animate emotions in a character’s performance
Animate all the features of your character’s face, including their eyes and tongue
You can use Audio2Face at runtime or in more traditional content creation pipelines. It offers various output formats and includes the ability to connect a custom blend shape mesh and export the resulting blend weights.
Minimum Mesh Requirements for Full Face Character Setup¶
A2F requires that a Head Mesh must be broken down into its individual mesh components. Head Mesh, Left Eye, Right Eye, Lower Teeth and Tongue must be individual meshes and cannot contain sub meshes. Please see Online Documentation and NVOD A2F Tutorial videos for further guidance.
In order to use Audio2Face, you must have:
Win 64 1909 or newer
Ubuntu Linux Version 20.04 or newer
If you’re trying to access sample assets in the local Omniverse mount, you need to install the Nucleus application from the Omniverse Launcher.
Interested in learning more?Open the Omniverse Audio2Face documentation portal