7. Replicator SceneBlox tutorial¶
The goal of this tutorial is to give a concrete example of generation with SceneBlox. The application is a toy case of building a labyrinth. The labyrinth consists of 4 kind of pieces : intersections, corridors, corners and dead-ends, which need to connect in a consistent way. The tile randomization spawns some obstacles in certain tiles.
The tutorial usd’s are stored in
For more details on sceneblox, checkout the manual here.
7.1. Predefined example¶
7.1.1. Running your first generation¶
Before running for the first time make sure to install dependencies using:
./python.sh -m pip install tools/scene_blox
If you want to see the generation in action without going into the details, try the pre-defined configuration files. From the repository root folder, to generate 5 labyrinths variants of size 9x9, run
./python.sh tools/scene_blox/src/scene_blox/generate_scene.py \ <omniverse://path/to/generation/folder/> \ --grid_config tools/scene_blox/parameters/labyrinth/rules.yaml \ --generation_config tools/scene_blox/parameters/labyrinth/generation.yaml \ --rows 9 --cols 9 --variants 5 \ --display \ --constraints_config tools/scene_blox/parameters/labyrinth/constraints.yaml \ --collisions
This will generate usd scenes in the given folder, and display the solving process of the grid.
7.1.2. Generation argument description¶
save_path: Folder where the scenes will be generated
–variants Number of variants of the scenes to be generated
–grid_config Path to the yaml containing the combination rules and tile size
–generation_config Path to the yaml containing generation configuration (base tile usd and randomization)
–constraints_config Path to the yaml with the initial grid constraints
–rows Number of rows for the generated grids, should be greater than 3 for best results
–cols Number of cols for the generated grids, should be greater than 3 for best results
–add_outer_boundary Add building border (specific for parking scenes)
–display Add a display showing the grid solving process
–collisions Check for collisions on objects generated that have both collisions and rigid body physics enabled
–units_in_meters Set the scene unit conversion (important for physics scene)
7.2. Detailed walkthrough¶
7.2.1. Defining the tiles¶
First, we need to define the basic of the tiles used: where they are stored and their size. This is done in tools/scene_blox/parameters/labyrinth/generation.yaml. This configuration file lists all tile types with their identifiers, where they are stored and their size.
tile_size: 5.0 fixed_prims: - prim_path: /Environment/Sky usd: /NVIDIA/Assets/Skies/Dynamic/CumulusHeavy.usd semantic: sky world_pose: position: [0, 0, 0] orientation: [90, 0, 0] cross: usd: /Isaac/Samples/Scene_Blox/Tutorial/cross.usd corridor: usd: /Isaac/Samples/Scene_Blox/Tutorial/corridor.usd corner: usd: /Isaac/Samples/Scene_Blox/Tutorial/corner.usd dead_end: usd: /Isaac/Samples/Scene_Blox/Tutorial/dead_end.usd
Note that here we also add a fixed sky, for lighting.
7.2.2. Defining the rules¶
First, we need to define the tile combination rules, which means to know which tile can be
next to which and with what orientation. This would be extremely tedious
and error-prone to list them by hand, so we provide example scenes and generate the
rules from there using a helper script
tools/scene_blox/src/scene_blox/rules_builder.py. We also do this is
several example scenes, one per tile type, then combine them with another helper script
A good method to follow is to leverage the fact that the rules are given by pairs. This means that once we have stated the rules for one tile type, it does not need to be mentioned in the next following scenes. This means that for example here, we can start by building an example scene with the rules for the intersection tile, then when building the example scene for corridor we do not need to include intersection and so on.
The scenes for the example, in the order they were built, are here :
Notice how the number of tile types decreases as we go, until there is only one tile type for the dead-ends.
To build an example scene for a tile type, the compatible tiles should be placed at the desired spacing (here 5m). If tiles are further apart than the tile size, they are not considered for building the rules. Add new tiles as references by using drag and drop, and do not modify the name as it is used to infer the tile types.
Once the scenes are built, generate the corresponding rules files by running
- usage: rules_builder.py [-h] [–rules_config RULES_CONFIG]
stage save_path tile_size
- positional arguments:
stage Path to the stage used as an example for rule generation save_path Path where the generated yaml will be saved tile_size Size of a tile (in scene units)
- optional arguments:
If not empty contains tile equivalence
All of the generated rules files can be found in
Once they are generated, they can be combined into one single rules configuration file
usage: rules_combiner.py [-h] [–config_files CONFIG_FILES [CONFIG_FILES …]] save_path
- positional arguments:
save_path Path to save the combined rules file.
- optional arguments:
–config_files CONFIG_FILES [CONFIG_FILES …] All files to be combined
This method can be long if there are many tile types, but it is the best to make sure that no rules are forgotten.
Now we have a minimum configuration to generate our labyrinths. In the next sections, we describe how to add constraints to the shape and how to add randomization to the tiles.
7.2.3. Constraining the labyrinth¶
The generated labyrinth are always consistent, but we might want to add some constraints to
the generated scenes. In this section, we illustrate how these constraints work. You can
tools/scene_blox/parameters/labyrinth/constraints.yaml for the full result, comment out sections if
you want to see the progressive effect of the constraints.
Let’s start by saying we want two corners of the labyrinth to be incoming corridors.
# Force corridor on (0, 0) - type: restrict_type identifiers: ["corridor"] area: rows: [[0, 0]] cols: [[0, 0]] - type: restrict_rotation identifier: ["corridor"] rotations:  area: rows: [[0, 0]] cols: [[0, 0]] # Force corridor on (-1, -1) - type: restrict_type identifiers: ["corridor"] area: rows: [[-1, -1]] cols: [[-1, -1]] - type: restrict_rotation identifier: ["corridor"] rotations:  area: rows: [[-1, -1]] cols: [[-1, -1]]
This restricts the two corners to one possibility, they will always be there in all successful generations
Now we want our labyrinth to be easy enough, so we restrict the number of dead ends to 4.
# No more than 4 dead ends - type: restrict_count identifiers: ["dead_end"] max_count:  area: rows: [[0, -1]] cols: [[0, -1]]
This will make sure that any generated labyrinth has at most 4 dead ends.
Finally, we might want to make the labyrinth less easy to escape, by making sure no tile on the border except the corner ones allow for an exit. To do so, we can constraint the border tiles rotations of corners and corridors to prevent outgoing tiles, and exclude intersections and dead ends.
# Prevent dead ends and crosses on the borders - type: exclude_type identifiers: ["dead_end", "cross"] area: rows: [[0, -1], [0, -1], [0, 0], [-1, -1]] cols: [[0, 0], [-1, -1], [0, -1], [0, -1]] # Prevent outgoing corridors - type: restrict_rotation identifier: ["corridor"] rotations: [1, 3] area: rows: [[0, 0]] cols: [[1, -2]] - type: restrict_rotation identifier: ["corridor"] rotations: [1, 3] area: rows: [[-1, -1]] cols: [[0, -2]] - type: restrict_rotation identifier: ["corridor"] rotations: [0, 2] area: rows: [[0, -1]] cols: [[0, 0]] - type: restrict_rotation identifier: ["corridor"] rotations: [0, 2] area: rows: [[0, -1]] cols: [[-1, -1]] # Prevent outgoing corners - type: restrict_rotation identifier: ["corner"] rotations: [2, 3] area: rows: [[0, -1]] cols: [[0, 0]] - type: restrict_rotation identifier: ["corner"] rotations: [0, 1] area: rows: [[0, -1]] cols: [[-1, -1]] - type: restrict_rotation identifier: ["corner"] rotations: [1, 2] area: rows: [[0, 0]] cols: [[0, -1]] - type: restrict_rotation identifier: ["corner"] rotations: [0, 3] area: rows: [[-1, -1]] cols: [[0, -1]]
This results in consistent labyrinths with exits only at the corners. Note that the generation handles only local constraints and consistency, so there is no guarantee that there is a path from one corner to another, or that there are no disconnected paths.
7.2.4. Adding randomization¶
Finally, we will add some randomization to the scene by spawning obstacles on some tiles.
The final configuration files for generation and for randomization are in
First, we spawn cones in the corridor tiles. We want to spawn up to 3 cones, each with
a probability of 0.33 to spawn. The configuration file is the following
root_prim: obstacles generated: - name: cone path: cone semantic: obstacle spawn_proba: 0.33 spawn_count: 3 usd_config: root: /Isaac/Environments/Simple_Warehouse/Props/ search_depth: 0 filter: S_TrafficCone.* position: base: [0, 0, 0] noise: type: normal params: mean: [0, 0, 0] stddev: [1.0, 0.25, 0] orientation: base: [0, 0, 0] noise: type: uniform params: low: [0, 0, -180] high: [0, 0, 180] scale: [0.01, 0.01, 0.01] physics: collision: convexHull rigid_body: true
Note that here we are enabling physics collision for the cones. We are also adjusting the scale of the asset, and adding noise both to the position and orientation. If collision check is enabled, the cones will not overlap with anything in collision when they are added to the scene, but if not, there is no guarantee.
Then, we want to add some obstacle piles to the dead ends (relative paths are obstacle_pile_1.yaml and obstacle_pile_2.yaml). Those piles are mutually exclusive, so in the generation file we split the selection between the two
dead_end: usd: /Isaac/Samples/Scene_Blox/Tutorial/dead_end.usd generation: - config: ["obstacle_pile_1.yaml", "obstacle_pile_2.yaml"] weights: [0.5, 0.5]
Note that while spawning the obstacle pile, we add collision to the different objects in the pile, and not the pile itself, by using the apply_children boolean.
root_prim: obstacles generated: - name: pile path: pile semantic: obstacle usd_config: root: /Isaac/Samples/Scene_Blox/Tutorial/ search_depth: 0 filter: obstacle_pile.usd position: base: [0, 0, 0] orientation: base: [0, 0, 0] physics: apply_children: true collision: convexHull
The final generation file is the following:
tile_size: 5.0 fixed_prims: - prim_path: /Environment/Sky usd: /NVIDIA/Assets/Skies/Dynamic/CumulusHeavy.usd semantic: sky world_pose: position: [0, 0, 0] orientation: [90, 0, 0] cross: usd: /Isaac/Samples/Scene_Blox/Tutorial/cross.usd generation: - config: hazards_corridors.yaml corridor: usd: /Isaac/Samples/Scene_Blox/Tutorial/corridor.usd generation: - config: hazards_corridors.yaml corner: usd: /Isaac/Samples/Scene_Blox/Tutorial/corner.usd generation: - config: ["None", "obstacle_pile_2.yaml"] weights: [0.7, 0.3] dead_end: usd: /Isaac/Samples/Scene_Blox/Tutorial/dead_end.usd generation: - config: ["obstacle_pile_1.yaml", "obstacle_pile_2.yaml"] weights: [0.5, 0.5]
Note how we reuse generation files from one tile to another. Also note the “None” configuration for the corners, so that it is possible to have corners without obstacles while still keeping a spawn probability of 1 for the pile in the configuration file itself.
In the end, we obtain labyrinths with a consistent setup, and with obstacle randomization.
7.3. Warehouse generation example¶
tools/scene_blox/parameters directory also contains a set of parameter files for warehouse generation.
./python.sh tools/scene_blox/src/scene_blox/generate_scene.py \ <omniverse://path/to/generation/folder/> \ --grid_config tools/scene_blox/parameters/warehouse/tile_config.yaml \ --generation_config tools/scene_blox/parameters/warehouse/tile_generation.yaml \ --cols 15 --rows 11 \ --constraints_config tools/scene_blox/parameters/warehouse/constraints.yaml \ --variants 1 --units_in_meters 1.0 --collisions
Which will result in the following output:
This generates full warehouses with closed-off extremities, containing a set of pre-defined cameras for observing the scenes, and which is generated using individual warehouse kit assets. The warehouse kit contains props like pallets, shelves etc, and also full building pieces that can be kitted together. The generation allows to pick building variants, and fill the building with a consistent set of assets: shelves of varying dimensions, pallets, piles of cardboard, cones and signs, guardrails etc. The constraints ensure that all shelves are aligned in the same direction (parallel to the longitudinal axis of the warehouse) for a more consistent aspect.
Note that the constraints are tuned for this specific number of rows and columns. The number of columns is constrained by the width of a the warehouse building and should always stay at 15 to ensure that the whole width is filled. The numbers of rows is constrained by the width of the warehouse building kit pieces, which is here tailored for two middle pieces and two end pieces, and should stay at 11 for a successful generation. The constraints file ensure that the middle pieces containing the actual building are spawned in the correct place only.
If it is necessary to change the length of the warehouse, take care of updating the constraints file to ensure that the middle and end pieces are spawned correctly. One of the end pieces is actually not part of the wavefunction collapse generation, but rather added at the end so that the generated warehouse can be left with one end open if desired (for better visualization).
It is highly recommended to generate these scenes with collision check enabled, as the scene is quite busy and could result in overlapping props. In some rare cases, some of the props could overlap and cause some unstable physics behavior. In that case, those props can be removed or modified by hand, or a new scene can be generated.