.. _isaac_sim_app_manual_replicator_composer: Replicator Composer Manual ========================== Replicator Composer is a tool for creating parameterizable offline datasets in Isaac Sim. This page explains how to write the input parameter file. * The :ref:`isaac_sim_app_manual_replicator_composer_parameter_list` provides the list of input parameters. * The :ref:`Replicator Composer Tutorial ` explains how to run Replicator Composer. Sections -------- * `Distributions`_ - Documentation for Distribution types. Parameters set to distributions enable random datasets. * `Profiles`_ - How to incorporate re-usable parameter sets. * `Groups`_ - How to add objects and lights to a scene using a parameter group. * `Asset Lists`_ - How to create an Asset List file. * `Paths`_ - Various rules for resolving paths in Replicator Composer. * `Units`_ - Units of parameters. * `Optimizing Dataset`_ - How to speed up dataset generation and reduce dataset size. Parameterization ---------------- Each parameter has one value type: number, string, bool, or tuple. A parameter can be set to one of the following: * A deterministic value: a primitive like ``5``, ``False``, or ``(255, 0, 0)`` * A stochastic value: a Distribution A stochastic value is defined as a Distribution which, when sampled, outputs a primitive value. Distributions are sampled during runtime. **Note, all parameter values are written in Python syntax.** Distributions ^^^^^^^^^^^^^ **Note, all distributions are using omni.replicator.core distributions, please see the docs there.** Uniform Distribution ******************** **Constructor(s)** .. code-block:: python Uniform(min_val: float, max_val: float) **Behavior** Returns a float between ``min_val`` and ``max_val``. **Example(s)** .. code-block:: yaml obj_scale: Uniform(0.5, 1.5) obj_color: Uniform((0, 0, 0), (1, 1, 1)) Normal Distribution ******************* **Constructor(s)** .. code-block:: python Normal(mean: float, var: float) **Behavior** Returns Gaussian (centered at ``mean`` with standard deviation ``std``). **Example(s)** .. code-block:: yaml obj_color: Normal((0.5, 0.5, 0.5), (0.1, 0.1, 0.1)) obj_vel: Normal((0, 0, 0), (100, 100, 100)) camera_coord: Normal((0, 0, 0), (500, 500, 500)) Range Distribution ****************** Note, Range Distribution is using omni.replicator.core.distributions.uniform. But for backward compatibility, we support the Range syntax. **Constructor(s)** .. code-block:: python Range(min_val: int, max_val: int) **Behavior** Returns an integer between ``min_val`` and ``max_val``. **Example(s)** .. code-block:: yaml obj_count: Range(0, 10) Choice Distribution ******************* **Constructor(s)** .. code-block:: python Choice(choices: List[Any], p(optional): List[float], with_replacements(optional): bool=True) **Behavior** | Returns an element from an element list, provided by an ``input_list``. | Elements are uniformly sampled or weighted by ``weights``, where ``weights`` will be normalized. | If ``with_replacements==True``, allow re-sampling the same element. If ``with_replacements==False``, each element can only be sampled once. **Example(s)** .. code-block:: yaml obj_color: Choice([(1, 0, 0), (0, 1, 0), (0, 0, 1)]) obj_model: Choice(["assets/models/shapes.txt"]) obj_model: Choice(["assets/models/warehouse.txt", "assets/models/hospital.txt", "/Isaac/Props/Forklift/forklift.usd"]) Walk Distribution ***************** **Constructor(s)** .. code-block:: python Walk(input_list: List[Any], ordered=True: bool) Walk(file: str, ordered=True: bool) **Behavior** Identical to a Choice, except elements are sampled w/o replacement until the ``input_list`` is empty. Once empty, the list is re-populated. * If :code:`ordered` is ``True``: elements in the list are sampled by walking from start to end. * If :code:`ordered` is ``False``: elements in the list are sampled randomly with removal. **Example(s)** .. code-block:: yaml obj_color: Walk("assets/colors/macbeth_chart.txt") camera_coord: Walk("assets/coords/camera_trajectory.txt") Profiles ^^^^^^^^ | In the Input Parameter file, the ``profiles`` key provides a list of parameter files that each supply a parameter set. | They can be used to create re-usable parameter sets that can be pointed to by different input parameter files. * The first (top) parameter file will override the parameters of the second parameter file, and so on. * The profile file :code:`parameters/profiles/default.yaml` is automatically set as the lowest parameter set (which provides all default parameter values). Note, :code:`parameters/profiles/default.yaml` is not suggested to be changed. * The input parameter file is automatically set as the highest parameter set. * Note, profiles defined within a profile parameter file will be ignored. * Profiles can define groups to create inheritable groups (see `Group Inheritance`_). The following :code:`input.yaml` contains three profiles: .. code-block:: python ... profiles: - parameters/profiles/hawk_camera.yaml # provides camera parameters tuned to the Hawk camera - parameters/profiles/obj_classes.yaml # provides useful inheritable object parameter groups - parameters/profiles/mtrepte_default.yaml # default parameter set tuned to mtrepte's preferences Which creates the following parameter set stack: .. code-block:: bash 1. input.yaml 2. hawk_camera.yaml 3. obj_classes.yaml 4. mtrepte_default.yaml 5. default.yaml Groups ^^^^^^ A group is a set of object and/or light parameters (parameters prefixed with **obj_*** or **light_***). A group is nested with a unique group name. Group names cannot be prefixed with **obj_** or **light_**. | For each scene, Replicator Composer will loop through each defined group to spawn in objects and lights. | **Therefore, in order to spawn in any objects or lights, there must be at least one group with a non-zero** obj_count **or** light_count. For instance, the 4 groups defined in the following :code:`input.yaml`. .. code-block:: yaml # flying cube group_a: obj_model: /Isaac/Props/Shapes/cube.usd obj_count: 1 # dropped warehouse objects group_b: obj_model: Choice(["assets/models/warehouse.txt"]) obj_count: Range(0, 20) obj_physics: True # colorful flying lights group_c: light_color: Uniform((0, 0, 0), (1, 1, 1)) light_count: Range(0, 5) # warm ceiling lights group_d: light_temp: 3000 light_temp_enabled: True light_count: 4 light_coord_camera_relative: False ... Group Inheritance ***************** | Similar to polymorphism in object-oriented programming, groups can be defined in terms of a hierarchical inherited group tree. | This is helpful for creating inheritable group parameter sets. For instance, the following :code:`profile.yaml` .. code-block:: yaml random_objects: obj_model: /Isaac/Props/Shapes/cube.usd obj_count: 0 # prevents objects of this group from spawning obj_rot: Uniform((0, 0, 0), (360, 360, 360)) obj_size: Uniform(100, 200) obj_reflectance: Uniform(0, 1) obj_metallicness: Uniform(0, 1) and :code:`input.yaml` .. code-block:: yaml random_flying_shapes: obj_model: Choice(["assets/models/shapes.txt"]) obj_count: Range(0, 10) inherit: random_objects random_falling_warehouse_objects: obj_model: Choice(["assets/models/warehouse.txt"]) obj_count: 5 obj_physics: True inherit: random_objects ... profiles: - parameters/profiles/profile.yaml Creates this object group polymorphism: .. image:: /content/images/isaac_replicator_composer_group_polymorphism.png :width: 500 .. _isaac_sim_app_manual_replicator_composer_manual_asset_lists: Asset Lists ^^^^^^^^^^^ Parameters set to Choice distributions can point to TXT files called Asset Lists, where each line provides an element in the element list of a Choice. These lists can contain colors, object model paths, object textures paths, camera coordinates, etc. For instance, the following :code:`input.yaml` .. code-block:: yaml random_shapes: obj_model: Choice(["assets/models/shapes.txt"]) obj_color: Choice(["assets/colors/macbeth_chart.txt", (1, 0, 0)]) obj_count: Range(0, 20) with :code:`models/shapes.txt` .. code-block:: yaml /Isaac/Props/Shapes/cube.usd /Isaac/Props/Shapes/sphere.usd /Isaac/Props/Shapes/cylinder.usd ... with :code:`colors/macbeth_chart.txt` .. code-block:: yaml (0.06, 0.55, 0.87) (0.48, 0.8, 0.87) (0.6, 0.33, 0.62) (0.54, 0.51, 0.84) (0.68, 0.56, 0.42) ... .. _isaac_sim_app_manual_replicator_composer_paths: Paths ^^^^^ There are four ways paths in the input parameterization and command line arguments are resolved. Nucleus Server Paths ******************** | If a path is the primitive value of a parameter satisfying ***_model** or ***_texture** or ***_material**, the path is resolved as :code:`nucleus_server + path`. | The path must be prefixed with :code:`/`. For instance, for the following :code:`input.yaml` file .. code-block:: yaml forklift_texture_randomized: obj_model: /Isaac/Props/Forklifts/forklift.usd obj_texture: Choice(["assets/textures/photos.txt"]) obj_count: 1 ... nucleus_server: localhost With this :code:`textures/photos.txt` file .. code-block:: yaml /Users/mtrepte/photos/gravel.png /Users/mtrepte/photos/basket_weave.png The paths resolve as follows .. code-block:: yaml omniverse://localhost/Isaac/Props/Forklifts/forklift.usd omniverse://localhost/Users/mtrepte/photos/gravel.png omniverse://localhost/Users/mtrepte/photos/basket_weave.png **Note, if nucleus fails to find the asset on nucleus server, it will treat the path as an absolute path and tries to find the asset locally.** Mounted Paths ************* Else if the path is prefixed by :code:`*`, the path is resolved as :code:` + `. The ```` is provided from the command line argument :code:`--mount`. For instance, for the following :code:`input.yaml` .. code-block:: yaml forklifts: obj_model: Choice(["*/assets/models/forklifts.txt"]) obj_count: Range(0, 5) With the cmd .. code-block:: console $ ./python.sh tools/composer/src/main.py \ --input */parameters/forklift_in_warehouse.yaml \ --mount /home/mtrepte/replicator-workspace Then paths resolve as follows .. code-block:: yaml /home/mtrepte/replicator-workspace/parameters/forklift_in_warehouse.yaml /home/mtrepte/replicator-workspace/assets/models/forklifts.txt Absolute Paths ************** Else if the path is prefixed by :code:`/`, the path is resolved as :code:`path` (absolute path). For instance, for the following :code:`input.yaml` file .. code-block:: yaml forklifts: obj_model: Choice("/home/mtrepte/replicator-workspace/assets/models/forklifts.txt") obj_count: Range(0, 5) The paths resolve as follows .. code-block:: yaml /home/mtrepte/replicator-workspace/assets/models/forklifts.txt Relative Paths ************** Else the path is relative to the Replicator Composer folder in the Isaac Sim source. The path is resolved as :code:`/tools/composer + ` For instance, for this :code:`input.yaml` .. code-block:: yaml texture_randomized_objects: obj_model: Choice("assets/models/office.txt") obj_texture: Choice("assets/textures/synthetic.txt) obj_count: Range(0, 5) Then the paths resolve as follows .. code-block:: yaml /tools/composer/assets/models/office.txt /tools/composer/assets/textures/synthetic.txt Units ^^^^^ The units of parameters that refer to physical lengths are in scene units (determined by **scene_meters_per_unit**, which defaults to 1, which means 1 meter contains 1 scene unit), except for **focal_length** [mm] and **horiz/vert}_aperture** [mm]. Depth maps are in **scene_meters_per_unit**. All rotation is in degrees. Optimizing Dataset ^^^^^^^^^^^^^^^^^^ Ways to speed-up generation and reduce storage footprint. Speed up Generation ******************* Loading many objects and textures slows generation. Reducing **obj_count** and material / texture complexity can help. .. Setting **path_tracing** to ``True`` and any non-zero **pause** slows generation. Also, setting **obj_physics** to ``True`` slows generation and reducing **physics_simulate_time** can help. Reduce Dataset Size ******************* Setting **groundtruth_stereo** to ``False`` and / or **groundtruth_visuals** to ``False`` can help. Also, disable any output types that are not needed.