4. OmniGraph

OmniGraph is Omniverse’s visual programming framework. It provides a graph framework that can easily connect functions from multiple systems inside Omniverse. It is also a compute framework that allows for highly customized nodes so that users can integrate their own functionality into Omniverse and automatically harness the efficient computation backend.

Inside Omniverse Isaac Sim, OmniGraph is the main engine behind the Replicators, ROS and ROS2 bridges, access to many sensors, controllers, external input/output devices, UI, and much more. The purpose of this tutorial is giving you a fast introduction into the world of visual programming via OmniGraph. We highly recommend you to checkout the much more in-depth documentation on OmniGraph, for it is a key component in Omniverse Kit.

4.1. Learning Objectives

This tutorial details build an action graph to control a robot in Omniverse Isaac Sim, specifically, the Jetbot. The goal of this tutorial is to give you a fast introduction to using Action Graphs in Omniverse Isaac Sim.

10-15 Minute Tutorial

4.2. Getting Started

Prerequisites

4.3. Setting up the stage

On a new stage, start by right clicking and selecting create -> Physics -> Ground Plane anywhere in the viewport. Next, use the content browser to navigate to Isaac/Robots/JetBot and click and drag jetbot.usd onto the stage. Position the jetbot just above the ground plane. When completed, the jetbot should be under /World/jetbot in the context tree and the stage should look similar to this

../_images/isaac_tutorial_omnigraph_jetbot.png

Note

Click play! You should see jetbot fall and land on the stage. Click stop before continuing.

4.4. Building the graph

Select Window -> Visual Scripting -> Action Graph from the dropdown menu at the top of the editor. The Graph Editor will appear in the same pane as the Content browser. Click New Action Graph to open an empty graph. Type controller in the search bar of the graph editor, and drag an Articulation Controller and a Differential Controller onto the graph.

The Articulation Controller applies driver commands (in the form of force, position, or velocity) to the specified joints of any prim with an articulation root. To tell the controller which robot it’s going to control, first select the Articulation Controller node in the graph and open up the property pane. You can either keep usePath checked and type in the path to the robot /World/jetbot in the robotPath, or you can uncheck usePath and then click Add Targets near the top of the pane for input:targetPrim, and select jetbot in the pop up window.

The Differential Controller computes drive commands for a two wheeled robot given some target linear and angular velocity. Like the Articulation Controller, it also needs to be configured. Select the Differential Controller node in the graph, and then in the properties pane, set the wheelDistance to 0.1125 and the and the wheelRadius to 0.03.

When finished, the properties of the controllers should look like the following

../_images/isaac_tutorial_omnigraph_ACproperties.png
../_images/isaac_tutorial_omnigraph_DCproperties.png

The Articulation Controller also needs to know which joints to articulate. It expects this information in the form of a list of tokens or index values. Each joint in a robot has a name and the jetbot has exactly two. Verify this by examining the jetbot in the stage context tree. Within /World/jetbot/chassis are two revolute physics joints named left_wheel_joint and right_wheel_joint.

../_images/isaac_tutorial_omnigraph_tree.png

Type token into the search bar of the graph editor and add two Constant Token nodes to the graph. Select one, and set it’s value to left_wheel_joint in the properties pane. Repeat this for the other constant token node, but set the value to right_wheel_joint. Type make array into the search bar of the graph editor and add a Make Array node to the graph. Select the Make Array node and set Array Size to 2 in the properties pane. Finally, connect the constant token nodes to A and B of the Make Array node, and then the output of that node to the Joint Names input of the Articulation Controller node.

Finally, search for playback in the search bar of the graph editor and add an On Playback Tick node to the graph. This node will emit an execution event for every frame, but only while the simulation is playing. Connect the Tick output of the On Playback Tick node to the Exec In input of both controller nodes. Connect the Velocity Command output of the differential controller to the Velocity Command input of the articulation controller. When completed, the graph should look similar to this…

../_images/isaac_tutorial_omnigraph_jetbot_minimal.png

Press the play button and select the Differential Controller Node in the graph. Click and drag on either the angular or linear velocity values in the properties pane to change it’s value (or just click and type in the desired value).

../_images/isaac_tutorial_omnigraph_jetbotVid.gif

As an exercise, explore the available OmniGraph nodes and try to setup a graph to control the jetbot with the keyboard! The graph below is an example graph for controlling the jetbot with a keyboard.

../_images/isaac_tutorial_omnigraph_full.png

4.5. Summary

This tutorial covered the following topics:

  1. Setting up a stage with a simple robot

  2. Using OmniGraph to construct simple interfaces to a robot

4.5.1. Further Learning

  1. More in-depth concepts in OmniGraph

  2. Examples for using OmniGraph to connect input devices: OmniGraph: Input Devices

  3. Examples for composing OmniGraph via Python scripting: OmniGraph: Python Scripting