5. Isaac Sim Workflows

5.1. Learning Objectives

This tutorial summarizes the different workflows for developing in Omniverse Isaac Sim and ways to interface with it with your existing systems. After this tutorial, you will understand the uses and benefits of each workflow and the best way to include Omniverse Isaac Sim in application.

10-15 Minute Tutorial

5.2. Getting Started

There are three main workflows when developing in Omniverse Isaac Sim: GUI, extensions, standalone Python. Here is a quick summary of the key features and their recommended usages.

GUI

  • Key features: Visual, intuitive, specialized tools for populating and simulating a virtual world.

  • Recommended usage: World building, assemble robots, attach sensors, initialize ROS bridges

Extensions

  • Key features: Run asynchronously to allow interactions with the stage, hot reloading to reflect changes immediately, adaptive physics step for real-time simulation

  • Recommended usage: Building interactive GUIs, custom application modules, real-time sensitive applications

Standalone Python

  • Key features: control over timing of physics and rendering steps, can be run in headless mode

  • Recommended usage: Large scale training for reinforcement learning, systematic world generation and modification

5.3. GUI

Omniverse Isaac Sim’s GUI interface features are the same ones used in Omniverse Create, an application that is dedicated for world-building. Create tools make it easy to assemble, illuminate, simulate, and render scenes large and small, therefore making it the ideal place to build your virtual worlds, assemble robots, examine physics.

If you completed the previous three Introductory tutorials of Isaac Sim Interface, Environment Setup, Add Simple Objects, then you’ve already had a taste of using the GUI. To learn more about how to leverage the GUI for your robotics application, continue to our GUI tutorial series, Assemble a Simple Robot.

5.4. Extensions

Extensions are the core building block of Omniverse Kit based applications. They are individually built application modules. All the tools used in Omniverse Isaac Sim are built as extensions. They can be used across different Omniverse applications by simply installing it in the Extensions Manager.

One main feature of this workflow is that the application runs asynchronously. This enables the extension applications to interact with the USD stage without blocking rendering and physics stepping. It also allows for hot reloading, so you can change the application code while Omniverse Isaac Sim is running and then see the reflected changes in your application after saving the file, without shutting down or restarting Omniverse Isaac Sim.

Most of the action in an extension is done via callbacks that are triggered with certain events, such as a physics or rendering step, stage events, ticks in time.

In the following GIF, a cube is added as part of Hello World and the sample is hot reloaded.

Developing extensions from scratch is more involved and considered an advanced feature, so Omniverse Isaac Sim provides a basic robotics application that can be inherited in Python to start with the basic functionality needed for any application (load world, reset world, and clear world), as covered in Hello World. However, to develop a custom modular library on top of Omniverse Isaac Sim, go to Extension Documentation for more information.

5.4.1. Running Your First Extension Application

  1. Start Omniverse Isaac Sim through the Launcher or terminal.

  2. Go to the top Menu Bar and click Isaac Examples > Multi-Robot > RoboParty.

  3. The window for the RoboParty example extension should now be visible in the workspace.

  4. Click the LOAD button to load the World.

  5. Once loaded in the viewport, click the Start Party button under Task Controls. You have now interacted with your first Extension Application.

  6. When you are finished experimenting with the extension, click File > New From Stage Template > Empty.

5.5. Standalone Application

In this workflow, Omniverse Isaac Sim is launched via a python script, inside which the rendering and physics are stepped manually, to guarantee that stepping only happens after ascertain the completion of a set of commands.

5.5.1. Running Your First Standalone Application

  1. Open the terminal.

  2. Navigate to the package path and run the follow_target_with_rmpflow.py script

    cd ~/.local/share/ov/pkg/isaac_sim-2022.1.1
    ./python.sh standalone_examples/api/omni.isaac.franka/follow_target_with_rmpflow.py
    
  3. Move the target prim by selecting it in the viewport so that Franka follows it.

../_images/isaac_sim_follow_target.gif

5.5.2. Using Jupyter to Develop a Standalone Application

Omniverse Isaac Sim can be launched as headless in this workflow (i.e. without a visible and updating GUI). It can also be launched via a Jupyter notebook (Linux only). In the clip below, Jupyter notebook live syncs to the same USD so that changes are reflected in another Isaac Sim process launched through the terminal or the launcher.

Hello World covers this workflow in detail.

../_images/isaac_sim_jupyter_workflow.gif

5.6. ROS Application

For more information about using Omniverse Isaac Sim with ROS, see the ROS tutorial series starting with Import and Drive TurtleBot3.

5.7. Summary

This tutorial covered the following topics:

  • The different workflows to develop an application in Omniverse Isaac Sim

  • Running an example using each workflow

5.7.1. Next Steps

Choose your own adventure and continue on to the next tutorial that suits your workflow:

5.7.2. Further Learning

For a more in-depth look into the concepts covered in this tutorial, please see the following reference materials: