AI Animal Explorer Extension

Overview

AI Animal Explorer is an Omniverse extension that enables creators to quickly prototype unique 3D animal meshes. This tool can help generate ideas for game or movie characters in real-time by allowing users quick access to a distribution of shapes for each part of the animal and direct control over their arrangement.

The animal creation process involves two stages. In the first stage, the user can explore different looks for each part of the animal (e.g. head, ears, etc) via sliders. These sliders interpolate through the latent space of diffusion probabilistic models which are capable of generating a diverse collection of point clouds for each animal part. The user can also scale, rotate and translate each animal part to place them in the desired location. A Deep Marching Tetrahedra (DMTet) model is used to synthesize a 3D mesh from the point cloud in the final stage or interactively during exploration to produce the final result.

Installation

AI Animal Explorer can be easily installed into an existing Omniverse application, such as Create, through the Extension Manager.

  1. Bring up the Extension Manager by selecting Window->Extensions.

  2. Search for “aitoybox”. Select the aitoybox.animal_explorer extension and click the Install button. (this may take a few minutes, depending on your connection speed)

  3. Enable the extension by clicking on the toggle icon to bring up the Animal Explorer main window.

Using AI Animal Explorer

UI Overview

Animal Explorer UI Legend

Ref #

Option

Result

1

Toggle Part

Create animal part if enabled or delete part when disabled

2

Part Slider

Drag slider to explore different looks for the animal part

3

Input Part Slider Value

Type in a value for the part slider or drag to increase or decrease the value

4

Preview Mesh

Generates mesh from point cloud

5

Discard

Discard point cloud and preview mesh created and start over

6

Bake Mesh

Saves generated mesh to new layer in “/World/Animals/ID<uuid>” and discards point cloud

Applying Transforms

After selecting an animal part, use the Move, Rotate, and Scale tools in the ToolBar to apply transformations. The ears and legs are mirrored on the Y-Z plane when either the left or the right part is moved. These parts also maintain the same scale but can be rotated independently.

Supporting Research

This extension is made possible by the works Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis by Tianchang Shen, Jun Gao, Kangxue Yin, Ming-Yu Liu, Sanja Fidler, Diffusion Probabilistic Models for 3D Point Cloud Generation by Shitong Luo, Wei Hu, and 3DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variations by Kangxue Yin, Jun Gao, Maria Shugrina, Sameh Khamis, Sanja Fidler