Synthetic Data Recorder

This extension enables offline data generation of various synthetic data sensor output locally from multiple viewports.

To open the extension go to Synthetic Data->Synthetic Data Recorder and the extension UI will become visible.

Synthetic Data Recorder UI Window

The extension supports offline data recording for six sensors - RGB, Depth, Semantic and Instance Segmentation, 2D Tight and Loose bounding box.

For Sensor Settings, corresponding to each of the six sensors, the following options can be enabled/disabled.

  • Status: Enables/disables a given sensor

  • Save Array: Given the sensor is enabled, saves the sensor output as a numpy array.

  • Colorize: Given the sensor is enabled, saves the sensor output as a colorized png image.

For Recorder Settings, below are the options that can be set.

  • Output Directory: The local output folder when generated data will be saved.

  • Render Mode: User can select the current render mode or specifically chose Ray tracing or Path tracing mode.

  • Samples per pixel: This applies when render mode is set to Path tracing. The noise in the output image decreases as the number of samples per pixel increases.

  • Capture period in seconds: This specifies the frequency of the synthetic data recorder. 0 means data is recorded every frame.

  • Number of worker threads: Number of threads involved in the offline synthetic data recording process.

  • Size of queue: Maximum size of queue to store and process synthetic data. If value of this field is less than or equal to zero, the queue size is infinite.

  • Record animation: Enables/disables timeline based animation recording.

Note

Higher queue size implies that the recorder will utilize more system memory. Therefore, the system may run out of memory when queue size is infinite. When a finite queue size is specified, you may observe that the FPS drops as the queue gets close to being full.

Once the various options are set, press Play in the editor followed by pressing Start Recording button to start offline synthetic data recording. The same button acts as a toggle between start and stop recording. You can reset the counter for the number of recorded synthetic data to 0 by pressing the Reset button. This ensures for future recording, the index of the output files start from 0.

The extension also supports synthetic data recording from multiple viewports. After creating a new viewport via Window->New Viewport Window, an additional Sensor Settings tab will show up corresponding to the newly created viewport.

Synthetic Data Recorder Multi Viewport UI Window

Synthetic Data Recorder Usage Example

Load the warehouse single shelve stage that comes pre-loaded with semantic information.

Isaac/Samples/Synthetic_Data/Stage/warehouse_with_sensors.usd

Press Shift+C and Shift+L to hide all camera and light widgets respectively so that they don’t show up in recorded synthetic data. Alternatively, go to Show/Hide UI icon and hide Cameras and Lights from Show By Type option.

Hide camera and light

Switch to the camera named RandomCamera and press Play in the editor. This camera randomly moves around the warehouse with its viewport pointing to the cart.

Switch camera to RandomCamera

Update the settings as specified below and press Start Recording.

Synthetic Data Recorder settings

The synthetic data recorded will be available in the local folder specified within the Output Directory option of Recorder Settings.

Synthetic Data Output Directory