2. Synthetic Data Recorder¶
2.1. Learning Objectives¶
This tutorial introduces the Synthetic Data Recorder for Omniverse Isaac Sim, a GUI extension for recording synthetic data with the possibility of using custom Replicator writers to record the data in any custom format.
15-20 Minute Tutorial
2.2. Getting Started¶
The tutorial uses the following stage as an example:
Isaac/Samples/Replicator/Stage/full_warehouse_worker_and_anim_cameras.usd

The example stage comes preloaded with semantic annotations, as well as multiple cameras, a few of them being animated to move around the scene when running the simulation. To create custom camera movement animations take a look at the Camera Animation Tutorial.
Note
When using other scenes make sure to add semantic annotations to the scene otherwise most annotators (semantic_segmentation, 3d_bounding_box, etc.) will not be able to produce data.
2.3. Basic Usage¶
The recorder is split into two main parts: the Writer
frame - containing sensor, data, and output parameters; and the Control
frame - containing the recording functionalities such as start, stop, pause, and parameters such as the number of frames to execute.

2.3.1. Writer Parameters¶
The Render Products frame creates a list of render product entries using the Add New Render Product
button. By default a new entry is added to the list using the active viewport camera as its camera path (see left figure). If however cameras are selected in the stage viewer, these are added to the render products list (see right figure).
The render products list can contain the same camera path multiple times, however each time with a different resolution. All the entry values (camera path or resolution) can also be manually edited in the input fields.

The Parameters frame gives the possibility to choose between the default built-in Replicator writer (BasicWriter) or to choose a custom writer. The default writer parameters (mostly annotators) can be selected from the checkbox list. As custom writers have unknown parameters, these can be provided by the user in form of a json file containing all the required parameters. The path to the json file can be added in the Parameters Path
input field.

The Output frame (left figure) contains the working directory path where the data will be saved together with the folder name used for the current recording. By selecting between Overwrite
, Increment
, or Timestamp
the folder name can be modified to avoid overwriting previous recordings. The Overwrite
option will write over any existing data in the folder, the Increment
option will increment the folder name by 1 if it already exists, and the Timestamp
option will add a timestamp to the folder name if it already exists. The recorder can also write to S3 buckets by providing the bucket name and having the credentials set up.
The Config frame (right figure) can load and save the GUI writer state as a json config file. By default the extension loads the previously used configuration state.

2.4. Control¶
The Control frame contains the recording functionalities such as start, stop, pause, and parameters such as the number of frames to execute. The Start
button will create a writer given the selected parameters and start the recording. The Stop
button will stop the recording and clear the writer. The Pause
button will pause the recording without clearing the writer, and the Resume
button will resume the recording. The Step
button will execute one frame of the recording. Preview
will load any Replicator randomizers and preview one step without recording. Step N
will execute N frames of the recording by calling the Step
function N times. The Number of frames
input field will set the number of frames to execute. If the value is set to 0, the recording will run until the Stop
button is pressed. The RTSubframes
field will set the number of subframes to execute per frame. This can be used if randomized materials are not loaded correctly of if shadow rendering artifacts are present due to objects being moved. The Reset Timeline
checkbox will reset the animation timeline to the beginning of the animation when the recording is started or stopped.

2.5. Custom Writer Example¶
In order to support custom data formats custom writer can be registered and loaded from the GUI. In this example, we register a custom writer called MyCustomWriter
using the Script Editor
and use it with the recorder.
MyCustomWriter
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | from omni.replicator.core import AnnotatorRegistry, BackendDispatch, Writer, WriterRegistry
class MyCustomWriter(Writer):
def __init__(
self,
output_dir,
rgb = False,
semantic_segmentation = False,
):
self.version = "0.0.1"
self.backend = BackendDispatch({"paths": {"out_dir": output_dir}})
if rgb:
self.annotators.append(AnnotatorRegistry.get_annotator("rgb"))
if semantic_segmentation:
self.annotators.append(AnnotatorRegistry.get_annotator("semantic_segmentation"))
self._frame_id = 0
def write(self, data):
if "rgb" in data:
filename = f"rgb_{self._frame_id}.png"
self.backend.write_image(filename, data["rgb"])
if "semantic_segmentation" in data:
filename = f"semantic_segmentation_{self._frame_id}.png"
self.backend.write_image(filename, data["semantic_segmentation"])
print(f"Frame {self._frame_id} written to {self.backend.output_dir}")
self._frame_id += 1
def on_final_frame(self):
print(f"Final frame {self._frame_id} reached")
self._frame_id = 0
WriterRegistry.register(MyCustomWriter)
|
my_params.json
1 2 3 | {
"rgb": true
}
|

2.6. Replicator Randomized Cameras¶
In order to take advantage of Replicator randomization techniques, these can be loaded using the Script Editor before starting the recorder to run scene randomizations during recording. In this example we create a randomized camera using Replicator API. This can be attached as a render product to the recorder and for each frame the camera will be randomized with the given parameters.
Randomized Camera
1 2 3 4 5 6 7 8 9 10 | import omni.replicator.core as rep
with rep.new_layer():
camera = rep.create.camera()
with rep.trigger.on_frame():
with camera:
rep.modify.pose(
position=rep.distribution.uniform((-5, 5, 1), (-1, 15, 5)),
look_at="/Root/SM_CardBoxA_3",
)
|

The following figure shows the steps by instructions on setting up a Replicator randomization and running it with the recorder.
