JetBot Sample Application

This sample will focus on NVIDIA JetBot, designed as a showcase example for NVIDIA Jetson Nano compute platform. Before you go and get yourself a JetBot, you might to check it out in simulation. This tour is written as if you are running the simulation locally on a workstation, but applies equally if you are running the remote client and container instead.

Getting Started

To run the samples, you will want to log in to an Omniverse Nucleus server. Depending on which version you have, this is done either through a dialog box automatically displayed in the center of the screen, or as a pop up option on the lower left corner of the screen under content tag, as shown in the image below. The dialog box in the image shows the server is ‘localhost’, which applies if you are connecting to a Nucleus server installed on your local workstation alongside Omniverse Isaac Sim. If instead you wish to connect to a Nucleus server on a remote machine, type in the IP address and port of the Nucleus server you configured during the installation. The Nucleus server holds all of the example assets for this tour, as well as other starter content that ships with Isaac Sim, so if you haven’t yet installed Omniverse Nucleus, you must do so before proceeding with the tour.

../_images/isaac_jetbot_login.png

Ref #

Item

Description

1

Content Browser

For more details, see Content Browser

2

Add New Connection Dialog

Click here to add your external Nucleus server.

3

Isaac Content

Open the Isaac folder to browse the Isaac Sim

Now that you are logged into the Nucleus server, you should see several folders appear in the Content panel at the bottom of your Omniverse Isaac Sim application window, as shown in the image below. Click on the folder labeled ‘Isaac’ in the tree view index on the left hand side of the Content panel, and you should then see a set of folders appear in the right-hand side of the panel, as shown in the image.

../_images/isaac_jetbot_content.png

Run the Sample

Next, look for the menu labeled Isaac Examples in the menu bar at the top of the Omniverse Isaac Sim application window. Once you find the Isaac Examples item in the menu bar, select Isaac Examples->Controlling->Input Devices->JetBot Keyboard as shown in the image below, and you should notice that a dialog box with two buttons and a block of text pops up. You can dock this new JetBot Keyboard panel by dragging it onto one of the other panels in the application window. On the JetBot Keyboard panel, press the top button to load the JetBot Keyboard scene.

../_images/isaac_jetbot_load.png

Ref #

Item

Description

1

Isaac Examples menu

For more details, see Samples Index

2

Jetbot Sample dialog

Defines the features of the sample and controls its operation.

3

Load Jetbot button

Select this button to load the USD assets for the sample.

Then as shown in the video below, press the Play button located on the vertical bar to the left of the Viewport to start the simulation loop. The JetBot should now respond to the w, s, a, and d keys on your keyboard. Press the Reset button on the panel to return the robot to the origin of the scene. When you are finished test driving the model robot, press ‘Stop’ button to return JetBot to its starting pose.

Layout Explained

Let’s get familiar with the layout of Omniverse Isaac Sim GUI, so that you can locate the various menus and tabs that we will be mentioning throughout the tutorials using the image below. We will use terminology from Universal Scene Description, the system Omniverse uses to composite assets for simulation, so please review this section.

For more details, please check out the Isaac Sim Interface section of the documentation.

../_images/isaac_jetbot_stage.png

Ref #

Item

Description

1

Viewport

In Isaac Sim the action you see in the Viewport[1] takes place on a Stage.

2

Stage Panel

The Stage [2] panel appears to the right of the Viewport pan Isaac Sim GUI.

3

USD Assets

The top-level view of the scene composited on the Stage, comprising objects for the World, physics, jetbot, and background. In this context you might hear JetBot and other objects on the Stage referred to as ‘USD assets’ or ‘USD models’

4

Play Button

When you press the Play [4] button, the simulation clock starts running and action begins on the Stage and is seen in the Viewport.

5

Render Settings

The render settings dialogue offers an easy and refined way of changing rendering options.

6

Properties Panel

Below the Stage panel in the image below, notice the panels labeled ‘Details’ and ‘PhysX Properties’. Together with the Viewport, the Stage and the Content panels, the Details and PhysX Properties panels are fundamental to Isaac Sim, and you should become familiar with them as you begin to develop your own simulations.

Rendering Settings

There are multiple lights and types of lights at use in the JetBot scene. The main light switches to turn off all types of lights are found on the Real Time Mode Settings below. You can bring up this panel by selecting the Render Settings tab as mentioned in the previous section.

../_images/isaac_jetbot_render_panel.png

Ref #

Item

Description

1

Render Mode

Use the Realtime setting for simulations.

2

Ray Tracing

This panel displays all of the Ray Tracing settings.

3

Direct Lighting

To turn off Direct Lights, such as Distant, Sphere or Dome lights that are placed on the Stage, use the checkbox in the Direct Light section of the panel.

4

Direct Lighting Checkbox

If you turn off Direct Lights using the checkbox, you should see JetBot in ambient light, also known as Indirect Diffuse Light.

5

Indirect Diffuse Lighting

This section controls all diffuse lighting options.

6

Intensity

If you set the Intensity of Indirect Diffuse Light to zero, you should now see an image like the one below.

From the information presented above, you can recreate the following image by clicking on the Rendering Settings tab in your UI. You will then uncheck the Direct Lighting checkbox. This should immediately disable all of the lights in the scene. Notice that the white lines on the floor are emissive materials,so they glow in the dark when all lights are switched off. The yellow outline of JetBot is visible because the JetBot model is selected on the Stage.

../_images/isaac_jetbot_no-light.png

Viewport Settings

If you are following this guide step-by-step, up until now you have been using the default Perspective view in the Viewport. We can change the view by going to the upper left corner of the viewport, click on “Perspective”. The dropdown menu shows that our default options other than “Perspective”, are “Top”, “Front”, and “Right”. Click on any of them to see the corresponding view.

../_images/isaac_jetbot_camera_view.png

There is also an option “Camera”. Once we open that up, we can see that our other option is the “jetbot camera”. This camera is part of JetBot asset, created when it was first converted from CAD to the USD format. It corresponds to the onboard camera sensor. We can change the viewport to show the view from this camera by selecting “jetbot camera”. We can see from the JetBot’s image that the camera is at the front of the robot and pointing slightly down. You may even see a grey camera sitting on top of the JetBot when you first loaded the robot. Hence, the new view from the onboard camera should be a few white lines on a blue background, because we would be looking directly at the ground. If you press “Play” and move the jetbot around, you can see the image changes corresponding to the robot’s movement.

The camera you created can be configured as to focal length and distance, aperture and other properties, as below. You can read more about about them in the camera docs here.

../_images/isaac_jetbot_camera-properties.jpg

Training Your JetBot

Take a look at This Sample on how to train you JetBot in Omniverse Isaac Sim.