Data Generator

Many machine learning techniques rely on images and ground truth labels for training, and synthetic data is a powerful tool to support such applications. The Omniverse Kaolin Data Generator extension uses RTX ray and path tracing to render massive image datasets from a collection of 3D data, while also exporting custom ground truth labels from a variety of sensors.

Interface Overview

Ref #	Option	Result
1	Root Directory	Selects directory from which to recursively search for valid .usd, .usda and .usdc files.
2	Presets	Saves or imports presets specifying data generation parameters.
3	Preview	Renders a single frame using the specified parameters and provides a preview of the enabled sensors.
4	Run	Runs data generation using the specified parameters and saves the output to the specified directory.
5	Demo	Opens a browser window to this documentation.
6	Preview	Displays a preview of the enabled sensors, when applicable.

Output Settings

Ref #	Option	Result
1	Output Directory	Selects directory where output data will be saved.
2	Renders per Scene	Specifies the number of renders to produce for each asset.
3	Sensors	Specifies the type(s) of ground truth data to save. See Sensors

Assets Settings

Ref #	Option	Result
1	Fix Bottom Elevation	Fixes the bottom of each asset to a common elevation.
2	Normalize	Normalizes the size of assets to fit within a unit cube.
3	Up Axis	Rotates the assets to accommodate either Y-Up and Z-Up orientations.

Camera Settings

Uniform Sampling

Ref #	Option	Result
1	Camera Mode: UniformSampling	Uniformly samples camera poses based on distance, elevation and azimuth parameters.
2	Look At Position	Selected a target point which the camera is pointed towards.
3	Distance	Specifies the randomization parameters of the camera’s distance to the Look-at position.
4	Elevation	Specifies the randomization parameters of the camera’s elevation, in degrees.
5	Azimuth	Specifies the randomization parameters of the camera’s azimuth, in degrees.

Trajectory - Spiral

Ref #	Option	Result
1	Camera Mode: Trajectory	The camera follows a fixed trajectory.
2	Trajectory Mode: Spiral	Specifies a spiral path for the camera to follow.
3	Distance	Distance of camera from object.
4	Elevation Range	Range of elevations that the spiral path will take, in degrees.
5	Number of Rotations	Number of complete rotations that the spiral path will take.

Trajectory - JSON

Ref #	Option	Result
1	Camera Mode: Trajectory	The camera follows a fixed trajectory.
2	Trajectory Mode: CustomJson	Specifies a specific trajectory for the camera to follow.
3	Json Path	Path to valid JSON file containing trajectory information (see Custom Trajectory).
4	View Trajectory	Produces a preview of the trajectory as a series of cones visible in the scene. The trajectory preview can be translated, rotated and scaled if needed.

Randomizers

The following randomizers are based on the randomizer prims included by default in Empty.usda. For advanced use cases, these randomizers can be moved, duplicated, removed or new randomizers can be added.

Ref #	Option	Result
1	Lighting	Controls the randomization parameters of dome lights placed in the scene.
2	Camera Light	Controls the randomization parameters of a disk light co-located with the camera.
3	Sun	Controls the randomization parameters of a distant light acting like a sun.
4	Materials	Controls the randomization parameters of a material applied to the asset.

Sensors

The Data Generator provides a number of sensors which can be used to save various types ground truth.

Sensor	Output File Format	Description
RGB	PNG (.png)	RGB image of the Low Dynamic Range render.
Depth	Numpy Array (.npy)	Depth value for each image pixel intersecting geometry. “Linear” mode returns radial linear depth (ie. distance from the camera position), while “Inverse Depth” returns inverse planar Z depth (ie. in clip space).
Normals	PNG (.png)	Surface normals, normalized between 0 and 255.
Semantic Segmentation	Numpy Array (.npy)	Semantic Segmentation, where each pixel of the image is assigned a semantic ID. An arbitrary semantic label is applied to the asset if none is provided.
Instance Segmentation	Numpy Array (.npy)	Instance Segmentation, where each pixel of the image is assigned an instance ID. Only assets with semantic labels are considered. An arbitrary semantic label is applied to the asset if none is provided.
BBox 2D Tight	Frame Metadata (.json)	2D Bounding Box tightly bounding the unoccluded (visible) pixels of assets with assigned semantic labels.
BBox 2D Loose	Frame Metadata (.json)	2D Bounding Box bounding the full extents (ie. visible and occluded) pixels of assets with assigned semantic labels.
BBox 3D	Frame Metadata (.json)	3D Bounding Box bounding the full extents (ie. visible and occluded) extents of assets with assigned semantic labels.
Pointcloud	USD (.usda)	Generates a full RGB pointcloud obtained by taking multiple samples around the asset.

Custom Trajectory

The Data Generator can be configured so that a camera follows a pre-determined path. This can be used to replicate real-world capture sessions with synthetic objects and lighting or to simulate arbitrary paths. The trajectory file must be valid JSON and follow the following format:

A trajectory file consists of a list of poses where each pose has the following attributes:

Label	Description
id	frame number
time	time stamp
q	rotation as a quaternion
t	translation

An example trajectory file is shown below:

trajectory_sample.json

   [
       {
           "id": 0,
           "time": 0.0,
           "q": [0.9588384493307851, -0.28395215809170277, 0.0, 0.0],
           "t": [0.0, -1.0, -1.0]
       },
       {
           "id": 1,
           "time": 0.03333,
           "q": [0.9588384493307851, -0.28395215809170277, 0.0, 0.0],
           "t": [0.0, -1.0, -0.98]
       },
       {
           "id": 2,
           "time": 0.06666,
           "q": [0.9588384493307851, -0.28395215809170277, 0.0, 0.0],
           "t": [0.0, -1.0, -0.96]
       }
   ]