13. Interfacing with NVIDIA Isaac ROS Visual SLAM GEM

Isaac ROS provides individual packages (GEMs) and complete pipelines (NITROS) which include image processing and computer vision functionality that has been highly optimized for NVIDIA GPUs and Jetson platforms. While it is not a requirement to use Omniverse Isaac Sim with Isaac ROS, the two systems are compatible for some GEMs. Here is an examples of using Isaac ROS Visual SLAM Gem in conjuncture with Omniverse Isaac Sim.

13.1. Getting Started

Isaac ROS Visual SLAM provides a ROS2 package that performs stereo visual simultaneous localization and mapping (VSLAM) and estimates stereo visual inertial odometry using the Isaac Elbrus GPU-accelerated library.

  • Launch Omniverse Isaac Sim, open Window -> Extensions, disable ROS bridge and enable ROS2 Humble bridge.

  • Follow the instructions from Tutorial with Isaac Sim to setup both the workspace for Isaac ROS VSLAM and Isaac Sim.

  • Work with the asset provided in omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.1/Isaac/Samples/ROS2/Scenario/carter_warehouse_apriltags_worker.usd

13.2. Multiple Robots

To perform VSLAM on multiple robots in the same scene, we need to place each robot in its own ROS domain. Follow the steps below to turn the current single-robot scene into a multi-robot scene.


Start with a newly opened stage. Do not press play until all modifications of the scene are finished.

  1. In the scene provided in omniverse://localhost/NVIDIA/Assets/Isaac/2022.2.1/Isaac/Samples/ROS2/Scenario/carter_warehouse_apriltags_worker.usd, first update the parameters of the existing robot on the stage with Step 6 and 7 in the instructions from Tutorial with Isaac Sim

  2. Add a second robot by duplicating the existing one. Right click on “Carter_ROS” on the stage tree, and click “Duplicate”. A new robot named “Carter_ROS_01” will be added to the stage. You can rename it by right click on the name and “Rename”.

  3. The newly created robot will be overlapping the original one upon creation. Move one of them to a different spot on the floor to avoid collisions between the two.

  4. Next we need to make sure each robot is inside its own domain. In the “Search” bar on top of the Stage tab, search for “context”. There should be four entries for context, two of which should be associated with “Carter_ROS” and two with “Carter_ROS_01”. Each robot has two action graphs that are used to publish all its ROS topics, one graph for camera and sensors, the other for action commands. Each graph has one node that is used to indicate the Domain ID. Hence four total graph nodes appeared in the search result.

  5. For each of the four entries, type in the Domain ID number in the Property Tab Omnigraph Node -> Inputs -> domain_id. Make sure all the Domain ID associated with the same robot is the same. You can open and visualize each of the graph inside the Action Graph editor (Window -> Visual Scripting -> Action Graph), and check how the domain context id is connected to the rest of the nodes.

  6. Once simulation starts, two viewports will be created for each robot, corresponding to the stereo camera on board of the Carter. (Do not start the simulation right now.) We need to update the viewport id number in the new robot to make sure the viewports created by it do not have conflicting names with the original robot. Search for “viewport_id” in the stage tree. Once again there should be four results, the left and right camera for two robots. They should all have distinct id numbers. Change them accordingly in the Property Tab -> Omnigraph Node -> Inputs -> Value.

  7. The scene now should have two robots sitting on the same factory floor, each ready to publish a series of topics in separate domain context. You can now press play to start the simulation. Four viewports should pop-up, representing the left and right views of the stereo cameras on the two robots.

  8. To run Isaac ROS VSLAM GEM for each of the robot, simply set the environment variable inside each of the terminals running the VSLAM scripts to match the robot you wish the VSLAM is performed on. Set the variable by source ROS_DOMAIN_ID=1, and you can check later by echo $ROS_DOMAIN_ID. This must be performed for every single terminal where ROS messages are sent or received.

  9. After setting the environment variable, the same steps from 9 and on in the Tutorial with Isaac Sim can be followed to perform VSLAM.

  10. One easy way to identify the VSLAM instance with its corresponding robot is to also display the RGB image from the cameras of the robots inside the rviz windows opened in Step 11 of Tutorial with Isaac Sim. To do so, click on the Add button on the bottom of the Display tab on the left side of the Rviz window, go to the By Topic tab in the pop-up window, and select /rgb_left. Repeat it for adding another image display for /rgb_right.


13.3. Multiple GPU


Multi-GPU setup is automatically enabled in Omniverse Isaac Sim, no special steps are needed if running on multiple GPUs.

13.4. Isaac ROS Packages

Follow these links to run Isaac ROS packages with Isaac Sim: