Setting Up Isaac Sim

Local Workstation Deployment

Local Requirements

Here are the hardware and software requirements for running Isaac Sim locally:

  1. Ubuntu 18.04 Operating System

  2. NVIDIA GPU (RTX 2070 or higher)

  3. NVIDIA GPU Driver (minimum version 440.59)

  4. Additional System Recommendations

Running Natively

This section shows you how to run the Isaac Sim container natively on Ubuntu.

See Local Requirements

Steps

  1. Run the command below to confirm your GPU driver version is 440.59 or later.

$ nvidia-smi
  1. Download the Isaac Sim package and extract it to a local folder.

  2. Go to the root of the extracted folder and run:

$ ./_build/linux-x86_64/release/isaac-sim.sh

Note

To run Isaac Sim natively in headless mode, run:

$ ./_build/linux-x86_64/release/isaac-sim-headless.sh
  1. See Nucleus Deployment to install the Nucleus Server.

  2. See Adding Samples Assets to add the Isaac Sim sample assets to a Nucleus Server.

  3. Proceed to Running Isaac Sim For the First Time

Running Windowed Container

This section shows you how to run the Isaac Sim container on local Ubuntu.

See Container Requirements

Steps

  1. Run the command below to confirm your GPU driver version is 440.59 or later.

$ nvidia-smi
  1. Using command line to login into NGC before downloading the Isaac Sim container.

$ sudo docker login nvcr.io
Username: $oauthtoken
Password: <Your NGC API Key>
WARNING! Your password will be stored unencrypted in /home/username/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded

Note

Get the your NGC API Key by clicking Generate API Key at https://ngc.nvidia.com/setup/api-key.

  1. Run the command below to enable xserver access:

$ xhost +local:
non-network local connections being added to access control list
  1. Run:

$ sudo docker run --entrypoint ./runapp.sh --gpus all -e "ACCEPT_EULA=Y" --rm  -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime:ro -e DISPLAY=unix${DISPLAY} nvcr.io/nvidia/isaac-sim:2020.1_ea

This will download and launch the Isaac Sim container in windowed mode.

Note

  1. See Nucleus Deployment to install the Nucleus Server.

  2. See Adding Samples Assets to add the Isaac Sim sample assets to a Nucleus Server.

  3. Proceed to Running Isaac Sim For the First Time

Remote Workstation Deployment

Container Requirements

Here are the hardware and software requirements for running Isaac Sim in a container:

  1. Ubuntu 18.04 Operating System

  2. NVIDIA GPU (RTX 2070 or higher)

  3. NVIDIA GPU Driver (minimum version 440.59)

  4. Docker CE 19.03

  5. NVIDIA Container Toolkit

# Add the package repositories
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
$ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

$ sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
$ sudo systemctl restart docker

Access Remote Ubuntu Workstation

This section shows you how to access a remote Ubuntu workstation.

Steps

  1. If you have access to the remote workstation physically, install an SSH server to allow remote access:

$ sudo apt update
$ sudo apt install openssh-server
  1. Run the command below to get the remote workstaion IP address:

$ ifconfig
  1. Run the command below to access the remote workstation:

$ ssh <remote_workstation_username>@<remote_workstation_ip_address>
<remote_workstation_username>@<remote_workstation_ip_address>'s password:
  1. Proceed to Running Headless Container

Running Headless Container

This section shows you how to run the Isaac Sim container on remote Ubuntu workstation. Below are the steps to take while you are SSH into the remote workstation or AWS instance.

See Container Requirements

Steps

  1. Run the command below to confirm your GPU driver version is 440.59 or later.

$ nvidia-smi
  1. Using command line to login into NGC before downloading the Isaac Sim container.

$ sudo docker login nvcr.io
Username: $oauthtoken
Password: <Your NGC API Key>
WARNING! Your password will be stored unencrypted in /home/username/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded

Note

Get the your NGC API Key by clicking Generate API Key at https://ngc.nvidia.com/setup/api-key.

  1. Run:

$ sudo docker run --gpus all -e "ACCEPT_EULA=Y" --rm -p 47995-48012:47995-48012/udp -p 47995-48012:47995-48012/tcp -p 49000-49007:49000-49007/tcp -p 49000-49007:49000-49007/udp nvcr.io/nvidia/isaac-sim:2020.1_ea

This will download and launch the Isaac Sim container in headless mode.

Note

By using the -e “ACCEPT_EULA=Y” flag, you are accepting the license agreement of the image found at NVIDIA Omniverse Pre-Release Evaluation License Agreement.

  1. Proceed to Omniverse Kit Remote Client to live-stream Isaac Sim remotely.

AWS Deployment

Note

Live-streaming to Isaac Sim on an AWS instance is not working for some cases. This user guide will be updated when the live-streaming feature is fixed. There are no issues with live-streaming Isaac Sim on remote workstation within the same network.

AWS Requirements

Here are the requirements for running Isaac Sim on Amazon Web Services(AWS):

  1. An Amazon Web Services(AWS) account that is able to launch an EC2 instance with GPU support.

  2. (Optional) Terraform config files can be used to easily deploy Isaac Sim to AWS.

    Download Terraform

    Download the Terraform Config

Running Headless Container on AWS

To launch AWS instances manually and run Isaac Sim, please follow these steps in this section.

Steps

  1. Create your Key Pair PEM file by following the 8 steps in this Key Pair Guide.

    To prevent permission errors when trying to SSH into the AWS instance:

    On Linux, remember to chmod 400 yourkey.pem as instructed in the link above.

    On Windows, right click on “yourkey.pem” file, Properties:

    • Go to Security tab, click “Advanced”

    • Remove inheritance

    • Set current user as owner of the file and full permissions to only that user.

  2. From your AWS EC2 Console, click Launch Instance, select Community AMIs, search for “Isaacsim-Ubuntu-18.04-GPU-2020-05-14”, press Select.

Note

The AMI may not be visible to Hong Kong, Africa Cape Town, Middle East Bahrain and Milan zones.

Select AMI
  1. Scroll down and pick a GPU instance (g4dn.4xlarge or higher). Click Next: Configure Instance Details at bottom right corner.

../_images/isaac_main_gpu_instance.png
  1. Make sure Auto-assign public IP is enabled.

../_images/isaac_main_auto_assign_ip.png
  1. Click Add Storage at the bottom right. Set the Root volume to size bigger than 8GB (like 32GB), else Docker won’t be able to extract the container from NGC due to insufficient storage space.

../_images/isaac_main_storage.png
  1. Click Next: Add Tags and click Next: Configure Security Group.

Click Add Rule, select All traffic and My IP so the Omniverse Kit Remote Client and Visual Studio Code can see this instance’s IP address. If you are running the Omniverse Kit Remote Client on another workstation, select the Custom source and enter the external IP address of that workstation.

../_images/isaac_main_all_traffic.png
  1. Click Review and Launch. Click Launch. Point to your existing PEM key or create a new PEM key (create a new one will save the key in your browser Download folder). Select Acknowledge.

Click Launch Instance, it’ll give you a link, something similar to i-0edc9523b2fff2e44, click that instance link.

../_images/isaac_main_launch.png

If you see “You have requested more vCPU capacity than your current vCPU limit”, you might need to stop your running instances to free up the vCPU usage and try again.

  1. Click Connect next to that Launch Instance button

../_images/isaac_main_connect_aws.png
  1. To SSH into the AWS instance:

As shown above, on Linux or Windows, run this command from where your pem key folder is, replace the ubuntu@ec2-54-67-42-74.us-west-1.compute.amazonaws.com with your instance:

$ ssh -i "yourkey.pem" ubuntu@ec2-54-67-42-74.us-west-1.compute.amazonaws.com

On Windows, you can also use Putty to SSH:

Convert your PEM key to Putty key.

See Connecting to Your Linux Instance from Windows Using PuTTY

Put your instance name above in the host name:

../_images/isaac_main_host_name_putty.png

Select SSH/Auth, point Browse to your converted Putty key. Then click Open and Yes, it should be able to SSH in now.

../_images/isaac_main_auth.png
  1. Using command line to login into NGC before downloading the Isaac Sim container.

$ sudo docker login nvcr.io
Username: $oauthtoken
Password: <Your NGC API Key>
WARNING! Your password will be stored unencrypted in /home/username/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded

Note

Get the your NGC API Key by clicking Generate API Key at https://ngc.nvidia.com/setup/api-key.

See Isaac Sim Container on NGC

  1. Run:

$ sudo docker run --gpus all -e "ACCEPT_EULA=Y" --rm -p 47995-48012:47995-48012/udp -p 47995-48012:47995-48012/tcp -p 49000-49007:49000-49007/tcp -p 49000-49007:49000-49007/udp nvcr.io/nvidia/isaac-sim:2020.1_ea

This will download and launch the Isaac Sim container in headless mode.

Note

By using the -e “ACCEPT_EULA=Y” flag, you are accepting the license agreement of the image found at NVIDIA Omniverse Pre-Release Evaluation License Agreement.

  1. Proceed to Omniverse Kit Remote Client to live-stream Isaac Sim remotely.

Using Terraform

Terraform is an infrastructure provisioning manager that helps users spin up AWS instances, download and launch Isaac Sim with minimal configuration.

Steps

  1. Download the Terraform Config and extract the package to your local folder.

  2. Go to the directory aws.

  3. Copy terraform.tfvars.template to terraform.tfvars.

  4. Edit terraform.tfvars with user configuration:

    • Client Name: The client name is a unique alpha-numeric string that groups the provisioned resources. It should be unique within your organization; if not, then conflicts may arise with your teammates’ setup. If you’re creating a server for yourself, then choose your LDAP. If sharing your server with your immediate team, then use your team’s name.

    • AWS Region: Location where instance(s) are launched. Choose a region closest to you in order to minimize latency. For now, one of {us-west-1, us-east-1}. Other regions will be made available as desired. Defaults to “us-west-1”.

    • AWS Access Key and Secret Key: Create an IAM admin user if you don’t have one already. This is the user that will be responsible for spinning up resources. Create an Access Key for the user, then copy and paste the values for the access key and associated secret here. See Appendix’s Creating AWS Access Key & Secret.(AWS admin privilege required).

    • SSH Key Name: An SSH Key is needed to SSH into the server instance. The build files will look for a public key file with the name “${ssh_key_name}.pub”. Copy the filename (without extension) here. Keep the files safe and share only with users that you allow to access your instances. See Appendix’s Creating an SSH Key.

    • NGC Secret Name: An NGC API key is required to pull the latest Docker image of Kit. See Appendix’s Creating a Secret for the NGC API Key (AWS admin privilege required).

    • Accept EULA: User to enter “yes” to accept EULA found at NVIDIA Omniverse Pre-Release Evaluation License Agreement.

  5. Download Terraform.

  6. Copy the Terraform executable to the aws folder.

  7. In your shell window, run these build commands from the aws folder.

    $ terraform init                # One-time initialization
    $ terraform plan                # Create build plan
    $ terraform apply               # Do the build
    
  • Wait for Terraform to launch the AWS instance, download the Isaac Sim container from NGC and launch it:

    • As the instance is starting up, you may see:

      pending_tasks_count = 1,
      running_tasks_count = 0
      
    • But over time, as the service gets up and running, you should see after the 3 minutes:

      pending_tasks_count = 0,
      running_tasks_count = 1
      
  • If Terraform is successful, then it should print out running_tasks_count” = 1 as shown below:

    Outputs:
    
    ecs_cluster = {
    "arn" = "arn:aws:ecs:us-east-1:363168841439:cluster/leo-ecs-cluster"
    "cluster_name" = "leo-ecs-cluster"
    "id" = "arn:aws:ecs:us-east-1:363168841439:cluster/leo-ecs-cluster"
    "pending_tasks_count" = 0
    "registered_container_instances_count" = 1
    "running_tasks_count" = 1
    "setting" = [
        {
        "name" = "containerInsights"
        "value" = "disabled"
        },
    ]
    "status" = "ACTIVE"
    }
    instances = {
    "i-081d02e0339e75b08" = "54.90.243.81"
    }
    

Under the “instances” block above, there is a key-value pair. The key string is the AWS assigned name of the machine instance.

The value is the public IP address of this instance. Please note down this IP address for the use in the next steps.

You could retrieve this IP address again with the apply command and try to ping it:

$ terraform apply               # Update and show current state
$ ping <public_ip_address>

If something goes wrong, clean up and try again:

$ terraform refresh             # Update state from AWS
$ terraform destroy             # Destroy all resources
$ terraform plan                # Re-create build plan
$ terraform apply               # Do the build
  1. Proceed to Omniverse Kit Remote Client to live-stream Isaac Sim in AWS remotely via it’s public IP address.

Notes:

  • If you do not see any pending or running tasks, the task has probably crashed. To resolve this, see Troubleshooting.

  • If you made changes to the deployment in AWS directly, this may cause issues with Terraform applying new settings or cleaning up. Some changes maybe resolved with a state refresh:

    $ terraform refresh             # Update state from AWS
    
  • To update deployment with changes to any .tf files:

    $ terraform refresh             # Update state from AWS
    $ terraform apply               # Update deployment
    

Omniverse Kit Remote Client

This is a streaming client to view Isaac Sim remotely on your desktop or workstation without a powerful GPU.

Remote Client Requirements

Ubuntu 18.04

Download the Omniverse Kit Remote Linux Client

Please install these packages to run the streaming client:

$ sudo apt-get install libavcodec57 libavformat57 libavutil55

Windows 10

Download and install the Omniverse Kit Remote Windows Client

The application is installed at this location:

C:\Program Files\NVIDIA Corporation\Omniverse\Kit Remote

Connect to Isaac Sim

Below are steps to live-stream a remote deployment of Isaac Sim to you local Linux or Windows workstation using Omniverse Kit Remote Client:

  1. Get the IP address of your Isaac Sim instance.

See AWS Deployment

On an AWS EC2 instance, see Getting IP Addresses of AWS EC2 Instance to get the public IP of the instance.

On a remote Linux workstation, run:

$ ifconfig
  1. Run Omniverse Kit Remote Client

On Linux:

$ ./omniverse-kit-remote.sh -s <remote_ip_address>

On Windows:

> omniverse-kit-remote.exe -s <remote_ip_address>

Note

  • Replace <remote_ip_address> above with your Isaac Sim instance Public IP.

  • To connect to a local headless instance of Isaac Sim, use IP 127.0.0.1.

  • Each Omniverse Kit instance can only connect to one Omniverse Kit Remote Client. Connecting to an Omniverse Kit instance that is currently serving a Omniverse Kit Remote Client will result in an error for the second user.

  • The Omniverse Kit Remote Client will create a window of resolution 1280x720 by default. It is recommended to set your display resolution to 1080p(1920x1080).

  • Errors will still be displayed in the logs even when running Omniverse Kit Remote Client is successful. This will be cleaned up later on. Right now, the logs would be useful for debugging.

  1. See Nucleus Deployment to install the Nucleus Server.

  2. See Adding Samples Assets to add the Isaac Sim sample assets to a Nucleus Server.

  3. Proceed to Running Isaac Sim For the First Time

Nucleus Deployment

To run the Isaac Sim Samples, it is required to connect to a Nucleus server with the sample assets. This section will guide you on installing the Nucleus server and uploading the Isaac Sim sample assets.

See Nucleus for more information.

Nucleus Requirements

See Nucleus API Specifications

Nucleus Installation

Option #1: Linux

Steps

  1. Download the Nucleus Linux Installer

  2. Run:

$ chmod +x omniverse-nucleus-2019.3A.2282-1-linux-installer.run
$ ./omniverse-nucleus-2019.3A.2282-1-linux-installer.run
...
Please type y to accept, n otherwise:
  1. To check the status of Nucleus, run:

$ systemctl status omniverse-server
● omniverse-server.service - Omniverse Server
Loaded: loaded (/lib/systemd/system/omniverse-server.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2020-06-04 10:02:00 CDT; 34s ago
Main PID: 27330 (omni.server.app)
    Tasks: 31 (limit: 4915)
CGroup: /system.slice/omniverse-server.service
        ├─27330 /opt/nvidia/omniverse/server/omni.server.app -input false
        └─27374 ./omni.delta-server.app -input false
  1. To access the Nucleus Web UI, go to http://localhost:8080/login.

  2. To access the Nucleus server from a local instance of Isaac Sim, use localhost:3009.

Option #2: AWS

To launch AWS instances manually and install Nucleus, please follow these steps in this section.

Steps

  1. Create your Key Pair PEM file by following the 8 steps in this Key Pair Guide.

    To prevent permission errors when trying to SSH into the AWS instance:

    On Linux, remember to chmod 400 yourkey.pem as instructed in the link above.

    On Windows, right click on “yourkey.pem” file, Properties:

    • Go to Security tab, click “Advanced”

    • Remove inheritance

    • Set current user as owner of the file and full permissions to only that user.

  2. From your AWS EC2 Console, click Launch Instance, select Community AMIs, search for “ubuntu 18.04”, press Select on the first AMI.

Note

The AMI might not be visible to Hong Kong and Bahrain zones.

Select AMI
  1. Scroll down and pick a CPU instance. Click Next: Configure Instance Details at bottom right corner.

../_images/isaac_main_cpu_instance.png
  1. Make sure Auto-assign public IP is enabled.

../_images/isaac_main_auto_assign_ip.png
  1. Click Add Storage at the bottom right. Set the Root volume to size bigger than 8GB (like 32GB), else Docker won’t be able to extract the container from NGC due to insufficient storage space.

../_images/isaac_main_storage.png
  1. Click Next: Add Tags and click Next: Configure Security Group.

  2. Click Add Rule, select All traffic and My IP so the Nucleus server can be access from the current machine.

../_images/isaac_main_all_traffic.png
  1. Click Add Rule, select All traffic and Custom and add the public IP address of the workstation running Isaac Sim. This could be the external IP address of a workstation or the public IP address of the AWS instance running Isaac Sim.

  2. Click Review and Launch. Click Launch. Point to your existing PEM key or create a new PEM key (create a new one will save the key in your browser Download folder). Select Acknowledge.

Click Launch Instance, it’ll give you a link, something similar to i-0edc9523b2fff2e44, click that instance link.

../_images/isaac_main_launch.png

If you see “You have requested more vCPU capacity than your current vCPU limit”, you might need to stop your running instances to free up the vCPU usage and try again.

  1. Click Connect next to that Launch Instance button

../_images/isaac_main_connect_aws.png
  1. To SSH into the AWS instance:

As shown above, on Linux, run this command from where your pem key folder is, replace the ubuntu@ec2-54-67-42-74.us-west-1.compute.amazonaws.com with your instance:

$ ssh -i "yourkey.pem" ubuntu@ec2-54-67-42-74.us-west-1.compute.amazonaws.com
  1. Download the Nucleus Linux Installer

  2. To copy the Nucleus installer to the AWS instance:

As shown above, on Linux, run this command from where your pem key folder is, replace the ubuntu@ec2-54-67-42-74.us-west-1.compute.amazonaws.com with your instance:

$ scp -r -i "yourkey.pem" "<path_to_downloaded_nucleus_installer>" ubuntu@ec2-54-67-42-74.us-west-1.compute.amazonaws.com:/home/ubuntu
  1. Next, SSH into the instance using the command in step 12.

  2. To install Nucleus as root run:

$ sudo -i
# cd /home/ubuntu
# chmod +x omniverse-nucleus-2019.3A.2282-1-linux-installer.run
# ./omniverse-nucleus-2019.3A.2282-1-linux-installer.run
...
Please type y to accept, n otherwise:
  1. To check the status of Nucleus, run:

$ systemctl status omniverse-server
● omniverse-server.service - Omniverse Server
Loaded: loaded (/lib/systemd/system/omniverse-server.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2020-06-04 10:02:00 CDT; 34s ago
Main PID: 27330 (omni.server.app)
    Tasks: 31 (limit: 4915)
CGroup: /system.slice/omniverse-server.service
        ├─27330 /opt/nvidia/omniverse/server/omni.server.app -input false
        └─27374 ./omni.delta-server.app -input false
  1. See Getting IP Addresses of AWS EC2 Instance to get the public and private IP of the instance.

  2. To access the Nucleus Web UI, go to http://<ip_address>:8080/login.

  3. To access the Nucleus server from Isaac Sim, use <ip_address>:3009. Try the private IP address of the Nucleus server if both the Isaac Sim and Nucleus instance is on the same region and VPC.

Option #3: Docker

This method is for advanced users. See Nucleus Docker

Adding Samples Assets

There are two methods to add the Isaac Sim sample assets to the Nucleus Server.

Method #1: Package

The first method is to download the asset package and manually upload it to a Nucleus server. This method allows editing to the files and it does not require the Nucleus server to have access the Internet.

Steps

  1. Download the Isaac Sim Asset Package.

  2. Extract the package to a temporary folder.

  3. Login to the Nucleus server via the Web UI as the omniverse user. See Web Service.

  4. Create a root folder on the Nucleus server and name it Isaac.

../_images/isaac_main_nucleus_create_folder.png
  1. Upload the extracted package to this Isaac folder.

../_images/isaac_main_nucleus_isaac_folder.png
  1. If the Isaac root folder could not be created in the Web UI, try creating it in Isaac Sim after connecting to the Nucleus server:

../_images/isaac_main_upload_assets.gif

Method #2: Mount

The second method is to create a mount on the Nucleus server. This method requires the Nucleus Server to have access to the Internet and no storage is used on the Nucleus server. The mounted folder is also read-only.

Steps

  1. Login to the Nucleus server via the Web UI as the omniverse user. See Web Service.

  2. Create a New mount on the root folder with the settings below:

Name: Isaac
Type: Amazon S3
Host: nvidia-omniverse-isaac.s3-us-west-1.amazonaws.com
Service: s3
Redirection: http://d30atda72sku6a.cloudfront.net
../_images/isaac_main_nucleus_add_mount.png

The folder should look like this:

../_images/isaac_main_nucleus_mount_folder.png

Editing Python Code

You can edit Isaac Sim Python code in real-time and view the changes. Any text editor can be used to edit the Python file but Visual Studio Code is preferred as it has an extension that allow accessing files on a remote workstation and easily modifying them.

Visual Studio Code Local Connection

Below are steps to access a local deployment of Isaac Sim:

  1. Download and install Visual Studio Code.

  2. The samples source files are at:

_build/linux-x86_64/release/exts/omni.isaac.samples/omni/isaac/samples/scripts

You can edit the source files (leonardo_preview.py). Save and see the changes hot-loaded immediately.

For example, change the label on the “Create Scenario” button to “Create My Scenario”:

self._create_franka_btn = self._window.layout.add_child(omni.kit.ui.Button("Create My Scenario"))

Save the code change with Ctrl + S and you should see the new button is hot-loaded and appear immediately without any recompilation.

../_images/isaac_main_hotloaded.png

Visual Studio Code Remote Connection

Preparation Steps

Below are the steps needed on the remote workstation or AWS EC2 instance where Isaac Sim was deployed:

  1. To get the docker container ID, run:

$ sudo docker ps
  1. Copy the sample folder from the running container to the host:

$ sudo docker cp <container_id>:/isaac-sim/_build/linux-x86_64/release/exts/omni.isaac.samples/omni/isaac/samples/ /var/lib/isaac-sim/
  1. Stop the docker container:

$ sudo docker stop <container_id>
  1. Make the samples folder on host writable:

$ sudo chmod -R 777 /var/lib/isaac-sim/scripts
  1. Re-run the container with:

$ sudo docker run --gpus all --rm -e "ACCEPT_EULA=Y" -v /var/lib/isaac-sim/data:/root/.local/share/omniverse/Kit/2020.1 -v /var/lib/isaac-sim/scripts:/isaac-sim/_build/linux-x86_64/release/exts/omni.isaac.samples/omni/isaac/samples/scripts:Z -p 47995-48012:47995-48012/udp -p 47995-48012:47995-48012/tcp -p 49000-49007:49000-49007/tcp -p 49000-49007:49000-49007/udp nvcr.io/nvidia/isaac-sim:2020.1_ea

Now, any changes to the /var/lib/isaac-sim/scripts folder will be synchronized with the docker container.

Edit Python Code Remotely

Below are the steps needed on the local workstation which will be accessing the remote deployment of Isaac Sim:

  1. Download and install Visual Studio Code.

  2. Install the Remote-SSH extension in Visual Studio Code.

    See here for more information on this extension.

  3. To SSH into the docker container, run SSH connection command (replace the <remote_ip_address> with AWS instance public IP or the IP address to the remote workstation in the previous section):

    $ ssh -i "<path_to_yourkey.pem>" <username>@<remote_ip_address>
    
  4. Run Visual Studio Code with the Remote-SSH extension above. Then click the bottom left button ><.

  5. Select Remote-SSH: Open Configuration File ….

  6. Select the first file (C:\Users\<youruser>.ssh\config).

  7. Add the following and replace <public_ip_address> with your IPv4 Public IP above:

    Host Remote Isaac Sim
        HostName <remote_ip_address>
        IdentityFile <path_to_yourkey.pem>
        User <username>
    
../_images/isaac_main_vscode_connect.png
  1. Save the config file above.

  2. Click that bottom left button again \ Connect to host \ Remote Isaac Sim.

You can open the Leonardo Preview source code folder (Ctrl + Shift + E or the 2 page button at top left cornerOpen Folder) at /var/lib/isaac-sim. Click OK.

../_images/isaac_main_open_folder.png
  1. The source files are at:

/var/lib/isaac-sim/samples

You can edit the source files (leonardo_preview.py). Save and see the changes hot-loaded immediately.

For example, change the label on the “Create Scenario” button to “Create My Scenario”:

self._create_franka_btn = self._window.layout.add_child(omni.kit.ui.Button("Create My Scenario"))

Save the code change with Ctrl + S and you should see the new button is hot-loaded and appear immediately without any recompilation.

../_images/isaac_main_hotloaded.png
  1. To check the source changes are persistent:

Use another terminal to SSH into your remote workstation or AWS instance. Run sudo docker ps to get the docker container ID

$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
65083dc18bc3 nvcr.io/nvidia/isaac-sim:2020.1_ea "/bin/sh -c 'service…" About an hour ago Up About an hour 0.0.0.0:47995-48012->47995-48012/tcp, 0.0.0.0:47995-48012->47995-48012/udp, 0.0.0.0:49000-49007->49000-49007/tcp, 0.0.0.0:49000-49007->49000-49007/udp, 0.0.0.0:5000->22/tcp unruffled_bose

To stop that docker, run this, replacing 65083dc18bc3 with your docker ID

$ sudo docker stop 65083dc18bc3

Run the docker command again and you should see your source changes (_create_another_franka_btn) still persist.

Troubleshooting

Reading the Logs in a Container

To ensure the Isaac Sim in a container is running, read the logs. Here’s how:

  1. If the Isaac Sim container is on a remote machine, SSH into the docker host using a terminal. Run this command from where your pem key folder is, replace the ubuntu@ec2-54-67-42-74.us-west-1.compute.amazonaws.com with your instance or remote host IP address:

    $ ssh -i "yourkey.pem" ubuntu@ec2-54-67-42-74.us-west-1.compute.amazonaws.com
    
  2. Access the running container by running:

    $ docker exec -it <container_id_or_name>
    $ cd isaac-sim/_build/target-deps/kit_sdk_release/_build/linux-x86_64/release/data/Kit/isaac-sim-headless/<version_number>
    $ tail -f omniverse-kit.log                    # To watch the log and auto refresh, CTRL-C to exit
    $ grep "Stream Server" omniverse-kit.log       # To check if livestream is ready
    
  3. Watch for these lines in the logs. Omniverse Kit is ready to connect to when you see these lines (after about 2 mins):

    [Info] [carb.livestream.plugin] Stream Server: Net Stream Server Instance Created
    [Info] [carb.livestream.plugin] Stream Server: streaming instance started - waiting for a client...
    

Getting IP Addresses of AWS EC2 Instance

To get the public and private IP addresses of an AWS EC2 instance, go to the Instances section of the EC2 Dashboard and select the instance. See the image below for an example of the Private and Public IPs:

../_images/isaac_main_aws_ip_address.png

SSH into the AWS EC2 Instance

If you need to directly access an AWS EC2 instance that was created from the deployment above, run these steps to SSH into the instance:

$ ssh -i "<ssh_key_name>.pem" ubuntu@<public_ip_address>

Restarting the Container

If you’ve set up the server with Terraform, then AWS will check the server container every few minutes and restart it if something went wrong.

If you didn’t use Terraform, or for some reason you want to manually restart the container, then run these steps from your shell:

  1. SSH into the AWS instance.

    $ ssh -i "<ssh_key_name>.pem" ubuntu@<public_ip_address>
    
  2. List all running containers and find the container ID for the Leonardo app.

$ sudo docker ps
CONTAINER ID        IMAGE
823686a7036d      nvcr.io/nvidia/isaac-sim...2020.1_ea
  1. Restart the container.

$ sudo docker restart [CONTAINER ID]
  1. View docker logs.

$ sudo docker logs [CONTAINER ID]

Setting up Docker

Once you have docker on Linux installed, follow the instructions at Post-installation steps for Linux to set it up so you would not need to use sudo to run a docker container.

Appendix

Isaac Sim Container on NGC

To download the Isaac Sim 2020.1 container go to:

Note

Please use your NVIDIA Developer Program credentials on the NVIDIA NGC website.

Creating AWS Access Key

Create an Access Key by following the instructions here:

Creating SSH Key

On Linux:

  1. Run:

    $ mkdir ~/.ssh
    $ chmod 700 ~/.ssh
    $ ssh-keygen -t rsa
    
  2. Enter your passphrase twice.

  3. Your public key is at .ssh/id_rsa.pub in your home folder and private key at .ssh/id_rsa.

  4. Save the public key to “aws” folder of the Terraform Config.

  5. Copy the filename (without extension) to the config field. Keep these files safe and share only with users that you allow to access your instances.

On Windows:

  1. Download PuTTYgen

  2. Launch PuTTYgen, and click on “Generate a public/private key pair”.

  3. Click on “Save public key” and name the file “${ssh_key_name}.pub”. This is your Public Key file.

  4. From the “Conversions” menu, select “Export OpenSSH key” and name the file “${ssh_key_name}.pem”. This is your Private Key file.

  5. Edit the properties of the “${ssh_key_name}.pem” file.

    • Go to security settings, click “Advanced”

    • Remove inheritance

    • Set current user as owner of the file and full permissions to only that user.

    • This is to prevent permission errors when trying to SSH into the instance.

  6. Save the public key to “aws” folder of the Terraform Config

  7. Copy the filename (without extension) to the config field. Keep these files safe and share only with users that you allow to access your instances.

Creating NGC API Key

If you don’t already have an API key:

  1. Generate your NGC API Key at:

https://ngc.nvidia.com/setup/api-key

  1. Go to AWS Secrets Manager:

https://us-west-1.console.aws.amazon.com/secretsmanager/home?region=us-west-1#/listSecrets

  1. Click Store a new secret.

  2. Select Other type of secrets.

  3. Set the following two sets of Secret key/value pairs

Key

Value

Username

$oauthtoken

Password

<Your_NGC_API Key>

  1. Give the secret a name

  2. Use the name in the Terraform Config file

Notes:

  • Multiple users can reuse the same secret.

  • Each secret is region dependent.