1.2. Isaac Sim Workflows

1.2.1. Learning Objectives

This tutorial summarizes the different workflows for developing in Omniverse Isaac Sim and ways to interface with it with your existing systems. After this tutorial, you will understand the uses and benefits of each workflow and the best way to include Omniverse Isaac Sim in application.

10-15 Minute Tutorial

1.2.2. Getting Started

There are three main workflows when developing in Omniverse Isaac Sim: GUI, extensions, standalone Python. Here is a quick summary of the key features and their recommended usages.

GUI

  • Key features: Visual, intuitive, specialized tools for populating and simulating a virtual world.

  • Recommended usage: World building, assemble robots, attach sensors, initialize ROS bridges

Extensions

  • Key features: Run asynchronously to allow interactions with the stage, hot reloading to reflect changes immediately, adaptive physics step for real-time simulation

  • Recommended usage: Building interactive GUIs, custom application modules, real-time sensitive applications

Standalone Python

  • Key features: control over timing of physics and rendering steps, can be run in headless mode

  • Recommended usage: Large scale training for reinforcement learning, systematic world generation and modification

1.2.3. GUI

Omniverse Isaac Sim’s GUI interface features are the same ones used in Omniverse USD Composer, an application that is dedicated for world-building. Create tools make it easy to assemble, illuminate, simulate, and render scenes large and small, therefore making it the ideal place to build your virtual worlds, assemble robots, examine physics.

If you completed the previous three Introductory tutorials of Isaac Sim Interface, Environment Setup, Add Simple Objects, then you’ve already had a taste of using the GUI. To learn more about how to leverage the GUI for your robotics application, continue to our GUI tutorial series, Assemble a Simple Robot.

1.2.4. Extensions

Extensions are the core building block of Omniverse Kit based applications. They are individually built application modules. All the tools used in Omniverse Isaac Sim are built as extensions. They can be used across different Omniverse applications by simply installing it in the Extensions Manager.

One main feature of this workflow is that the application runs asynchronously. This enables the extension applications to interact with the USD stage without blocking rendering and physics stepping. It also allows for hot reloading, so you can change the application code while Omniverse Isaac Sim is running and then see the reflected changes in your application after saving the file, without shutting down or restarting Omniverse Isaac Sim. Most of the action in an extension is done via callbacks that are triggered with certain events, such as a physics or rendering step, stage events, ticks in time.

1.2.4.1. Writing A Custom Python Extension

The easiest way to get started using the Extension workflow is to use the Omniverse Isaac Sim Extension Template Generator to populate a simple UI-based extension on your local machine. The available extension templates give a useful starting point for many Omniverse Isaac Sim applications and are structured to educate the user in building a custom UI tool that meets their needs.

Note

For more information on the structure and implementation details of the custom extensions, check out the Isaac Sim Extension Template tutorial.

To create and enable a new extension using the Extension Template Generator, follow these steps or reference the video below:

  1. Create a folder on your computer where you want Omniverse Isaac Sim to search for user extensions.

  2. Navigate to Isaac Utils -> Generate Extension Templates in the toolbar.

  3. Select the type of template that you want to start with, and fill in the required fields.

  4. Navigate to Window -> Extensions in the toolbar to open the Extensions Manager.

  5. Click the hamburger icon in the Extensions Manager, and then “Settings” in the sub-menu to add the path to the folder you created for your user extensions.

  6. Find your extension in the Extensions Manager under “Third Party Extensions” and enable it. It will now appear in the toolbar.

An alternative way to enable extensions is to add command-line arguments when running Omniverse Isaac Sim from the terminal: isaac_sim.sh –ext-folder {path_to_user_ext_folder} –enable {ext_directory_name}

Each extension template is meant to be self-explanatory. You can quickly get familiar with the template code by reading the README.md file in the provided Python module.

1.2.4.2. Writing A Custom Cpp Extension

If you want to write your own extension containing C++ code, Omniverse Kit provide an extension template as well as examples of its usage. Go to Kit C++ Extension Template for detailed instructions.

1.2.5. Standalone Application

In this workflow, Omniverse Isaac Sim is launched via a python script, inside which the rendering and physics are stepped manually, to guarantee that stepping only happens after ascertain the completion of a set of commands.

1.2.5.1. Running Your First Standalone Application

  1. Open the terminal.

  2. Navigate to the package path and run the follow_target_with_rmpflow.py script

    ./python.sh standalone_examples/api/omni.isaac.franka/follow_target_with_rmpflow.py

  3. Move the target prim by selecting it in the viewport so that Franka follows it.

../_images/isaac_sim_follow_target.gif

1.2.5.2. Using Jupyter to Develop a Standalone Application

Omniverse Isaac Sim can be launched as headless in this workflow (i.e. without a visible and updating GUI). It can also be launched via a Jupyter notebook (Linux only). In the clip below, Jupyter notebook live syncs to the same USD so that changes are reflected in another Isaac Sim process launched through the terminal or the launcher.

Hello World covers this workflow in detail.

../_images/isaac_sim_jupyter_workflow.gif

1.2.6. ROS Application

For more information about using Omniverse Isaac Sim with ROS, see the ROS tutorial series starting with URDF Import: Turtlebot.

1.2.7. Summary

This tutorial covered the following topics:

  • The different workflows to develop an application in Omniverse Isaac Sim

  • Running an example using each workflow

1.2.7.1. Next Steps

Choose your own adventure and continue on to the next tutorial that suits your workflow:

1.2.7.2. Further Learning

For a more in-depth look into the concepts covered in this tutorial, please see the following reference materials: