4.1. Hello World

NVIDIA Omniverse™ Kit, the toolkit that Omniverse Isaac Sim uses to build its applications, provides a Python interpreter for scripting. This means every single GUI command, as well as many additional functions are available as Python APIs. However, the learning curve for interfacing with Omniverse Kit using Pixar’s USD Python API is steep and steps are frequently tedious. Therefore we’ve provided a set of APIs that are designed to be used in robotics applications, APIs that abstract away the complexity of USD APIs and merge multiple steps into one for frequently performed tasks.

In this tutorial, we will present the concepts of Core APIs and how to use them. We will start with adding a cube to an empty stage, and we’ll build upon it to create a scene with multiple robots executing mulitple tasks simultaneously, as seen below.

4.1.1. Learning Objectives

This tutorial series introduces the Core API. After this tutorial, you will learn

  • Creating a World and Scene as defined by the Core API.

  • How to add a rigid body to the Stage and simulate it using python in Omniverse Isaac Sim.

  • The difference between running python in an Extension Workflow vs a Standalone Workflow, and in Jupyter Notebook.

10-15 Minute Tutorial

4.1.2. Getting Started

Prerequisites

Begin by opening the Hello World example. Go to the top Menu Bar and Click Isaac Examples > Hello World.

The window for the Hello World example extension should now be visible in the workspace. Click the Open Source Code button to launch the source code for editing in Visual Studio Code.

Click the Open Containing Folder button to open the directory containing the example files. This folder contains three files: hello_world.py, hello_world_extension.py, and __init__.py.

The hello_world.py script is where the logic of the application will be added, while the UI elements of the application will be added in hello_world_extension.py script and thus linked to the logic.

  1. Click the LOAD button to load the World.

  2. click File > New From Stage Template > Empty to create a new stage, click Don’t Save when prompted to save the current stage.

  3. Click the LOAD button to load the World again.

  4. Open hello_world.py and press “Ctrl+S” to use the hot-reload feature. You will notice that the menu disappears from the workspace (because it was restarted).

  5. Open the example menu again and click the LOAD button.

Now you can begin adding to this example.

4.1.3. Code Overview

This example inherits from BaseSample, which is a boilerplate extension application that sets up the basics for every robotics extension application. The following are a few examples of the actions BaseSample performs:

  1. Loading the world with its corresponding assets using a button

  2. Clearing the world when a new stage is created.

  3. Resetting the world’s objects to their default states

  4. Handling hot reloading

World is the core class that enables you to interact with the simulator in an easy and modular way. It handles many time-related events such as adding callbacks, stepping physics, resetting the scene, adding tasks (this will be covered later in Adding a Manipulator Robot), etc.

A world contains an instance of a Scene. The Scene class manages simulation assets of interest in the USD Stage. It provides an easy API to add, manipulate, inspect, and reset different USD assets in the stage.

 1from omni.isaac.examples.base_sample import BaseSample #boiler plate of a robotics extension application
 2
 3class HelloWorld(BaseSample):
 4    def __init__(self) -> None:
 5        super().__init__()
 6        return
 7
 8    # This function is called to setup the assets in the scene for the first time
 9    # Class variables should not be assigned here, since this function is not called
10    # after a hot-reload, its only called to load the world starting from an EMPTY stage
11    def setup_scene(self):
12        # A world is defined in the BaseSample, can be accessed everywhere EXCEPT __init__
13        world = self.get_world()
14        world.scene.add_default_ground_plane() # adds a default ground plane to the scene
15        return

4.1.3.1. Singleton World

World is a Singleton, which means only one World can exist while running Omniverse Isaac Sim. The code below demonstrates how to retrieve the current instance of the World across different files and extensions.

 1from omni.isaac.examples.base_sample import BaseSample
 2from omni.isaac.core import World
 3
 4class HelloWorld(BaseSample):
 5    def __init__(self) -> None:
 6        super().__init__()
 7        return
 8
 9    def setup_scene(self):
10        world = World.instance()
11        world.scene.add_default_ground_plane()
12        return

4.1.4. Adding to the Scene

Next, use the Python API to add a cube as a rigid body to the scene.

 1from omni.isaac.examples.base_sample import BaseSample
 2import numpy as np
 3# Can be used to create a new cube or to point to an already existing cube in stage.
 4from omni.isaac.core.objects import DynamicCuboid
 5
 6class HelloWorld(BaseSample):
 7    def __init__(self) -> None:
 8        super().__init__()
 9        return
10
11    def setup_scene(self):
12        world = self.get_world()
13        world.scene.add_default_ground_plane()
14        fancy_cube = world.scene.add(
15            DynamicCuboid(
16                prim_path="/World/random_cube", # The prim path of the cube in the USD stage
17                name="fancy_cube", # The unique name used to retrieve the object from the scene later on
18                position=np.array([0, 0, 1.0]), # Using the current stage units which is in meters by default.
19                scale=np.array([0.5015, 0.5015, 0.5015]), # most arguments accept mainly numpy arrays.
20                color=np.array([0, 0, 1.0]), # RGB channels, going from 0-1
21            ))
22        return

  1. Press Ctrl+S to save the code and hot-reload Omniverse Isaac Sim.

  2. Open the menu again.

  3. click File > New From Stage Template > Empty, then the LOAD button. You need to perform this action if you change anything in the setup_scene. Otherwise, you only need to press the LOAD button.

  4. Press the PLAY button to start simulating the dynamic cube and see it falling.

Note

Every time the code is edited/changed, press “Ctrl+S” to save the code and hot-reload Omniverse Isaac Sim.

4.1.5. Inspecting Object Properties

Next, print the world pose and velocity of the cube. The highlighted lines show how you can get the objects using the name and query their properties.

 1from omni.isaac.examples.base_sample import BaseSample
 2import numpy as np
 3from omni.isaac.core.objects import DynamicCuboid
 4
 5class HelloWorld(BaseSample):
 6    def __init__(self) -> None:
 7        super().__init__()
 8        return
 9
10    def setup_scene(self):
11        world = self.get_world()
12        world.scene.add_default_ground_plane()
13        fancy_cube = world.scene.add(
14            DynamicCuboid(
15                prim_path="/World/random_cube",
16                name="fancy_cube",
17                position=np.array([0, 0, 1.0]),
18                scale=np.array([0.5015, 0.5015, 0.5015]),
19                color=np.array([0, 0, 1.0]),
20            ))
21        return
22
23    # Here we assign the class's variables
24    # this function is called after load button is pressed
25    # regardless starting from an empty stage or not
26    # this is called after setup_scene and after
27    # one physics time step to propagate appropriate
28    # physics handles which are needed to retrieve
29    # many physical properties of the different objects
30    async def setup_post_load(self):
31        self._world = self.get_world()
32        self._cube = self._world.scene.get_object("fancy_cube")
33        position, orientation = self._cube.get_world_pose()
34        linear_velocity = self._cube.get_linear_velocity()
35        # will be shown on terminal
36        print("Cube position is : " + str(position))
37        print("Cube's orientation is : " + str(orientation))
38        print("Cube's linear velocity is : " + str(linear_velocity))
39        return

4.1.5.1. Continuously Inspecting the Object Properties during Simulation

Next, print the world pose and velocity of the cube during simulation at every physics step executed. As mentioned in Isaac Sim Workflows, in this workflow the application is running asynchronously and can’t control when to step physics. However, you can add callbacks to ensure certain things happen before certain events.

Add a physics callback as follows.

 1from omni.isaac.examples.base_sample import BaseSample
 2import numpy as np
 3from omni.isaac.core.objects import DynamicCuboid
 4
 5class HelloWorld(BaseSample):
 6    def __init__(self) -> None:
 7        super().__init__()
 8        return
 9
10    def setup_scene(self):
11        world = self.get_world()
12        world.scene.add_default_ground_plane()
13        fancy_cube = world.scene.add(
14            DynamicCuboid(
15                prim_path="/World/random_cube",
16                name="fancy_cube",
17                position=np.array([0, 0, 1.0]),
18                scale=np.array([0.5015, 0.5015, 0.5015]),
19                color=np.array([0, 0, 1.0]),
20            ))
21        return
22
23    async def setup_post_load(self):
24        self._world = self.get_world()
25        self._cube = self._world.scene.get_object("fancy_cube")
26        self._world.add_physics_callback("sim_step", callback_fn=self.print_cube_info) #callback names have to be unique
27        return
28
29    # here we define the physics callback to be called before each physics step, all physics callbacks must take
30    # step_size as an argument
31    def print_cube_info(self, step_size):
32        position, orientation = self._cube.get_world_pose()
33        linear_velocity = self._cube.get_linear_velocity()
34        # will be shown on terminal
35        print("Cube position is : " + str(position))
36        print("Cube's orientation is : " + str(orientation))
37        print("Cube's linear velocity is : " + str(linear_velocity))

4.1.6. Adding a New Example in the Menu

So far, you have been editing the Hello World example. Next, you will create a new example under the Isaac Examples menu.

  1. Copy the current files to the user_examples folder under omni/isaac/examples

    cd omni/isaac/examples
cp hello_world/hello_world* user_examples/

  2. Edit the omni/isaac/examples/user_examples/__init__.py file by adding the following lines.

    1from omni.isaac.examples.user_examples.hello_world import HelloWorld
2from omni.isaac.examples.user_examples.hello_world_extension import HelloWorldExtension

  3. Edit the omni/isaac/examples/user_examples/hello_world_extension.py file as follows.

     1import os
 2from omni.isaac.examples.base_sample import BaseSampleExtension
 3from omni.isaac.examples.user_examples import HelloWorld
 4
 5
 6class HelloWorldExtension(BaseSampleExtension):
 7    def on_startup(self, ext_id: str):
 8        super().on_startup(ext_id)
 9        super().start_extension(
10            menu_name="",
11            submenu_name="",
12            name="Awesome Example",
13            title="My Awesome Example",
14            doc_link="https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/tutorial_core_hello_world.html",
15            overview="This Example introduces the user on how to do cool stuff with Isaac Sim through scripting in asynchronous mode.",
16            file_path=os.path.abspath(__file__),
17            sample=HelloWorld(),
18        )
19        return

Note

Every time the code is edited/changed, Press Ctrl+S to save the code and hot-reload Omniverse Isaac Sim.

4.1.7. Converting the Example to a Standalone Application

As mentioned in Isaac Sim Workflows, in this workflow, the robotics application is started when launched from Python right away, and you can control when to step physics and rendering.

  1. Open a new my_application.py file and add the following.

     1#launch Isaac Sim before any other imports
 2#default first two lines in any standalone application
 3from omni.isaac.kit import SimulationApp
 4simulation_app = SimulationApp({"headless": False}) # we can also run as headless.
 5
 6from omni.isaac.core import World
 7from omni.isaac.core.objects import DynamicCuboid
 8import numpy as np
 9
10world = World()
11world.scene.add_default_ground_plane()
12fancy_cube =  world.scene.add(
13    DynamicCuboid(
14        prim_path="/World/random_cube",
15        name="fancy_cube",
16        position=np.array([0, 0, 1.0]),
17        scale=np.array([0.5015, 0.5015, 0.5015]),
18        color=np.array([0, 0, 1.0]),
19    ))
20# Resetting the world needs to be called before querying anything related to an articulation specifically.
21# Its recommended to always do a reset after adding your assets, for physics handles to be propagated properly
22world.reset()
23for i in range(500):
24    position, orientation = fancy_cube.get_world_pose()
25    linear_velocity = fancy_cube.get_linear_velocity()
26    # will be shown on terminal
27    print("Cube position is : " + str(position))
28    print("Cube's orientation is : " + str(orientation))
29    print("Cube's linear velocity is : " + str(linear_velocity))
30    # we have control over stepping physics and rendering in this workflow
31    # things run in sync
32    world.step(render=True) # execute one physics step and one rendering step
33
34simulation_app.close() # close Isaac Sim

  2. Run it using ./python.sh /omni/isaac/examples/user_examples/my_application.py.

4.1.8. Converting the example to a Standalone Application using Jupyter Notebook

  1. Open a Jupyter notebook.

    ./jupyter_notebook.sh standalone_examples/notebooks/hello_world.ipynb

  2. Access the notebook through the browser as shown on the terminal.

  3. Execute the first cell in the notebook.

    Note

    Set the livesync_usd path to a proper value as indicated in the comments.

    1import getpass
2user = getpass.getuser()
3from omni.isaac.kit import SimulationApp
4# Set the path below to your desired nucleus server
5# Make sure you installed a local nucleus server before this
6simulation_app = SimulationApp({"livesync_usd": f'omniverse://localhost/Users/{user}/temp_jupyter_stage.usd'})

  4. Open the same USD file after launching Isaac Sim through the launcher or the terminal and turn the live syncing option on in the Layer tab

    Warning

    DON’T press play, pause, or stop in the GUI while in this workflow.

    ../_images/isaac_sim_live_sync.gif

  5. Run through the rest of the cells in the notebook, while viewing the stage directly in the Omniverse Isaac Sim GUI.

Note

A render call is needed at the end of each cell to propagate the changes to the synced USD (i.e. World.render or SimulationContext.render)

Note

Use world.clear() or the clear_stage function (from the omni.isaac.core.utils.stage module) at the beginning of a cell for iterative development.

4.1.9. Summary

This tutorial covered the following topics:

  1. Overview of the World and Scene classes.

  2. Adding content to the Scene via Python.

  3. Adding callbacks.

  4. Accessing dynamic properties for objects

  5. The main differences in a standalone application

  6. The main differences when developing an application using jupyter

4.1.9.1. Next Steps

Continue to Hello Robot to learn how to add a robot to the simulation.

Note

The next tutorials will be developed mainly using the extensions application workflow. However, conversion to other workflows should be straightforward given what was covered in this tutorial.