OmniGraph: Pick-and-Place Controller Node

Learning Objectives

This tutorial shows how to use Omniverse Isaac Sim OmniGraph Pick-and-Place Controller node to build a pick and place task via visual scripting. After this tutorial, you will better understand OmniGraph visual scripting principles, including how to Read Prim Attributes, Make Arrays, and actuate robots with included controller nodes.

10-15 Minute Tutorial

Getting Started

Prerequisites

  • Review the GUI Tutorial series, specifically OmniGraph, prior to beginning this tutorial.

Adding the USD Assets

Begin by adding a Franka manipulator and two cubes to the scene:

  1. Add the Franka asset from the Create > Isaac > Robots > Franka menu.

  2. Add two cubes from the Create > Shape > Cube.

  3. Resize as needed, and rename to PickingCube and PlacingCube.

  4. Move the PickingCube and PlacingCube to a desired picking and placing location within reach of the robot.

Adding Cubes to Scene

Omnigraph Setup

Now lets set up the OmniGraph to make the robot execute a Pick-and-Place action:

  1. Create the new action graph by navigating to Window > Visual Scripting > Action Graph.

  2. By default, the Action Graph loads into the bottom group on panels. Reposition the Action Graph and viewport panels as you like.

  3. In the Action Graph panel, select New Action Graph add a new graph to the simulation.

  4. Now add the following nodes to the graph:

  • On Playback Tick node executes the graph nodes every simulation timestep.

  • Isaac Pick-and-Place Controller node articulate the robot.

  • Make Array node to hold the pick-and-place event timings.

Adding Picking & Placing Position inputs

Note

You can “Quick Save” your Visual Scripting Layout by pressing Ctrl+7 hotkey, and then “Quick Load” it by pressing Ctrl+8. This is helpful for bouncing between Default and Visual Scripting layouts.

In the Make Array node, let’s add 10 entries that make the pick-and-place event timing

  1. Set the array type to double[].

  2. Press the + button 9 more times to make an array of size 10. Verify the size is 10 in the Property Panel.

  3. We suggest starting with the following time deltas: [0.01, 0.01, 1.0, 0.01, 0.01, 0.01, 0.01, 0.05, 0.01, 0.08]

  4. Connect to the Events DT input.

Adding Picking & Placing Position inputs

Omnigraph: Reading Attributes

Now let’s get the PickingCube and PlacingCube positions:

  1. From the Stage, drag and drop PickingCube onto the ActionGraph.

  2. Select Read Attribute from the pop-up window to create a new Read Attribute node.

  3. Navigate to the Attribute dropdown in the properties panel and select xformOp::translate.

  4. Connect the output of PickingCube’s Read Attribute node to the Picking Position input.

  5. Repeat this process for the PlacingCube, connecting its output to the Placing Position input.

Adding Picking & Placing Position inputs

Omnigraph: Pick-and-Place Controller Node

Lastly, let’s finalize the Pick-and-Place Controller node’s properties:

  1. Select the node and input “Franka” for its Robot Model property.

  2. Input “/Franka” for its Robot Prim Path property.

  3. Click usePrimPath so the controller can find its prim via the user input path.

Setting final properties of the Pick-and-Place Controller node

Now we should be ready to simulate. Verify that you have connected the nodes similar to this:

Pick-and-Place Controller Action Graph nodes

Now hit the SPACEBAR or Play button on the GUI. If set up correctly, the Franka robot should move towards the PickingCube, close its gripper, then move towards the PlacingCube, and finally open its gripper.

Summary

In this tutorial, we introduced the Isaac Pick-and-Place Controller Node and its functionality. We also detailed different OmniGraph input techniques.

Further Reading

To learn more about Omniverse Isaac Sim OmniGraph, see the other advanced tutorials in this series:

  1. OmniGraph: Input Devices

  2. OmniGraph: Imu Sensor Node

  3. OmniGraph: Python Scripting