Semantic Schema Editor

The functioning of segmentation and bounding box sensor data relies a lot on the semantic meaning of different objects in the simulation environment. The Semantic Schema Editor tool allows user to assign semantic labels to those objects.

Go to Synthetic Data->Semantic Schema Editor to enable it. There are two ways to assign semantic labels to objects.

../_images/isaac_camera_semantic_schema_editor.png

Apply semantic data on selected objects

As the title suggests, select a group of objects and then for each object, we specify the Type field as class and the Data field as the semantic label.

Apply semantic data on entire stage

We use a heuristic based approach to assign semantic labels to different objects in the scene on pressing Generate Labels button. The field Prim types to label specifies which USD object type will be assigned a semantic label. The field Class list specifies a list of class names. If an object in the scene contains any of these class names, then its semantic label will be assigned to that class name.

Semantic Schema Editor Usage Example

Load the simple room stage from here - Isaac/Environments/Simple_Room/simple_room.usd

This stage doesn’t come with any pre-loaded semantic information. We can verify this using the Synthetic Data Visualizer tool. For both scenarios below, click on visualizer icon in the Viewport window, check RGB and Semantic Segmentation sensors and press Visualize. This will show output from both sensors. The Semantic Segmentation output is red (represents background class) which signifies that the assets don’t contain semantic information.

Apply semantic data on selected objects

After enabling the Semantic Schema Editor extension, select the table (prim location is /Root/table_low_327/table_low and prim type is Mesh) and then in Semantic Schema Editor UI, specify the Type field as class and the Data field as table. This will apply semantic label to the table asset. We can verify using two approaches.

  1. Use the Visualize Synthetic Data tool, check RGB and Semantic Segmentation sensors and press Visualize. The Semantic Segmentation output is red except the table section of the image which is orange in color.

  2. Select /Root/table_low_327/table_low prim and go to Details tab. The attributes semantic:Semantics:params:semanticData and semantic:Semantics:params:semanticType will be set to table and class respectively.

Apply semantic data on entire stage

After enabling the Semantic Schema Editor extension, specify the Prim types to label field as Mesh, the Class list field as table,floor,wall and press Generate Labels. For all assets in the scene, it will check if the path of an asset of type Mesh contains any of the class names specified in the Class list field and then its semantic label will be assigned to that class name. We can verify using two approaches.

  1. Use the Visualize Synthetic Data tool, check RGB and Semantic Segmentation sensors and press Visualize. The Semantic Segmentation output will show different colors corresponding to different classes.

  2. We can also check attributes for some of the assets. For example, select /Root/table_low_327/table_low prim and go to Details tab. The attributes semantic:Semantics:params:semanticData and semantic:Semantics:params:semanticType will be set to table and class respectively. Similarly, select /Root/Towel_Room01_floor_bottom_218/Towel_Room01_floor_bottom prim and go to Details tab. The attributes semantic:Semantics:params:semanticData and semantic:Semantics:params:semanticType will be set to floor and class respectively.