.. _kaolin_data-generator: ================================== Data Generator ================================== Many machine learning techniques rely on images and ground truth labels for training, and synthetic data is a powerful tool to support such applications. The Omniverse Kaolin Data Generator extension uses RTX ray and path tracing to render massive image datasets from a collection of 3D data, while also exporting custom ground truth labels from a variety of sensors. Interface Overview ################## .. image:: /content/images/kaolin_data-generator_ui.jpg ========== ================================== ==================================================================== Ref # Option Result ========== ================================== ==================================================================== 1 Root Directory | Selects directory from which to recursively search for valid `.usd`, `.usda` and `.usdc` files. 2 Presets | Saves or imports presets specifying data generation parameters. 3 Preview | Renders a single frame using the specified parameters and provides a preview of the enabled sensors. 4 Run | Runs data generation using the specified parameters and saves the output to the specified directory. 5 Demo | Opens a browser window to this documentation. 6 Preview | Displays a preview of the enabled sensors, when applicable. ========== ================================== ==================================================================== .. _kaolin_data-generator_settings_outputs: Output Settings ############### .. image:: /content/images/kaolin_data-generator_ui_outputs.png ========== ================================== ==================================================================== Ref # Option Result ========== ================================== ==================================================================== 1 Output Directory | Selects directory where output data will be saved. 2 Renders per Scene | Specifies the number of renders to produce for each asset. 3 Sensors | Specifies the type(s) of ground truth data to save. See :ref:`kaolin_data-generator_sensors` ========== ================================== ==================================================================== .. _kaolin_data-generator_settings_assets: Assets Settings ############### .. image:: /content/images/kaolin_data-generator_ui_assets.png ========== ================================== ==================================================================== Ref # Option Result ========== ================================== ==================================================================== 1 Fix Bottom Elevation | Fixes the bottom of each asset to a common elevation. 2 Normalize | Normalizes the size of assets to fit within a unit cube. 3 Up Axis | Rotates the assets to accommodate either Y-Up and Z-Up orientations. ========== ================================== ==================================================================== .. _kaolin_data-generator_settings_camera-uniform: Camera Settings ############### Uniform Sampling ******************* .. image:: /content/images/kaolin_data-generator_ui_camera_uniform.png ========== ================================== ==================================================================== Ref # Option Result ========== ================================== ==================================================================== 1 Camera Mode: UniformSampling | Uniformly samples camera poses based on distance, elevation and azimuth parameters. 2 Look At Position | Selected a target point which the camera is pointed towards. 3 Distance | Specifies the randomization parameters of the camera's distance to the Look-at position. 4 Elevation | Specifies the randomization parameters of the camera's elevation, in degrees. 5 Azimuth | Specifies the randomization parameters of the camera's azimuth, in degrees. ========== ================================== ==================================================================== .. _kaolin_data-generator_settings_camera-trajectory-spiral: Trajectory - Spiral ******************* .. image:: /content/images/kaolin_data-generator_ui_camera_spiral.png ========== ================================== ==================================================================== Ref # Option Result ========== ================================== ==================================================================== 1 Camera Mode: Trajectory | The camera follows a fixed trajectory. 2 Trajectory Mode: Spiral | Specifies a spiral path for the camera to follow. 3 Distance | Distance of camera from object. 4 Elevation Range | Range of elevations that the spiral path will take, in degrees. 5 Number of Rotations | Number of complete rotations that the spiral path will take. ========== ================================== ==================================================================== .. _kaolin_data-generator_settings_camera-trajectory-json: Trajectory - JSON ******************* .. image:: /content/images/kaolin_data-generator_ui_camera_trajectory.png ========== ================================== ==================================================================== Ref # Option Result ========== ================================== ==================================================================== 1 Camera Mode: Trajectory | The camera follows a fixed trajectory. 2 Trajectory Mode: CustomJson | Specifies a specific trajectory for the camera to follow. 3 Json Path | Path to valid JSON file containing trajectory information (see :ref:`kaolin_data-generator_custom-trajectory`). 4 View Trajectory | Produces a preview of the trajectory as a series of cones visible in the scene. The trajectory preview can be translated, rotated and scaled if needed. ========== ================================== ==================================================================== .. _kaolin_data-generator_settings_randomizers: Randomizers ########### The following randomizers are based on the randomizer prims included by default in `Empty.usda`. For advanced use cases, these randomizers can be moved, duplicated, removed or new randomizers can be added. .. image:: /content/images/kaolin_data-generator_ui_randomizers.png ========== ================================== ==================================================================== Ref # Option Result ========== ================================== ==================================================================== 1 Lighting | Controls the randomization parameters of dome lights placed in the scene. 2 Camera Light | Controls the randomization parameters of a disk light co-located with the camera. 3 Sun | Controls the randomization parameters of a distant light acting like a sun. 4 Materials | Controls the randomization parameters of a material applied to the asset. ========== ================================== ==================================================================== .. _kaolin_data-generator_sensors: Sensors ####### The Data Generator provides a number of sensors which can be used to save various types ground truth. ================================== ====================== ==================================================================== Sensor Output File Format Description ================================== ====================== ==================================================================== RGB PNG (.png) | RGB image of the Low Dynamic Range render. Depth Numpy Array (.npy) | Depth value for each image pixel intersecting geometry. "Linear" mode returns radial linear depth (ie. distance from the camera position), while "Inverse Depth" returns inverse planar Z depth (ie. in clip space). Normals PNG (.png) | Surface normals, normalized between 0 and 255. Semantic Segmentation Numpy Array (.npy) | Semantic Segmentation, where each pixel of the image is assigned a semantic ID. An arbitrary semantic label is applied to the asset if none is provided. Instance Segmentation Numpy Array (.npy) | Instance Segmentation, where each pixel of the image is assigned an instance ID. Only assets with semantic labels are considered. An arbitrary semantic label is applied to the asset if none is provided. BBox 2D Tight Frame Metadata (.json) | 2D Bounding Box tightly bounding the unoccluded (visible) pixels of assets with assigned semantic labels. BBox 2D Loose Frame Metadata (.json) | 2D Bounding Box bounding the full extents (ie. visible and occluded) pixels of assets with assigned semantic labels. BBox 3D Frame Metadata (.json) | 3D Bounding Box bounding the full extents (ie. visible and occluded) extents of assets with assigned semantic labels. Pointcloud USD (.usda) | Generates a full RGB pointcloud obtained by taking multiple samples around the asset. ================================== ====================== ==================================================================== .. _kaolin_data-generator_custom-trajectory: Custom Trajectory ################## The :ref:`kaolin_data-generator` can be configured so that a camera follows a pre-determined path. This can be used to replicate real-world capture sessions with synthetic objects and lighting or to simulate arbitrary paths. The trajectory file must be valid JSON and follow the following format: A trajectory file consists of a list of poses where each pose has the following attributes: ======= ============================== Label Description ======= ============================== id frame number time time stamp q rotation as a quaternion t translation ======= ============================== An example trajectory file is shown below: .. code-block:: json :caption: trajectory_sample.json [ { "id": 0, "time": 0.0, "q": [0.9588384493307851, -0.28395215809170277, 0.0, 0.0], "t": [0.0, -1.0, -1.0] }, { "id": 1, "time": 0.03333, "q": [0.9588384493307851, -0.28395215809170277, 0.0, 0.0], "t": [0.0, -1.0, -0.98] }, { "id": 2, "time": 0.06666, "q": [0.9588384493307851, -0.28395215809170277, 0.0, 0.0], "t": [0.0, -1.0, -0.96] } ]