Native Workstation Deployment

Requirements

  1. Here are the requirements for running Omniverse Isaac Sim natively:

  2. Here are the requirements for running Omniverse Nucleus natively:

  3. See Nucleus Workstation to install Omniverse Nucleus.

  4. See Adding Sample Assets to add the Omniverse Isaac Sim sample assets to Nucleus.

  5. See Cache Workstation to install Omniverse Cache.

Isaac Sim on Omniverse Launcher

Omniverse Isaac Sim for Workstation can be downloaded through Omniverse Launcher.

It can be found and downloaded in the ‘Exchange’ tab. To simplify the process, enter isaac sim in the search bar.

../_images/isaac_main_launcher_exchange.png

When the download is complete, to the ‘Library’ tab and select ‘Isaac Sim’ on the sidebar. To run Isaac Sim, click the ‘Launch’ button.

../_images/isaac_main_launcher_library.png

Isaac Sim Modes

Isaac Sim App Launcher

This is a mini windowed app that will help launch any of the modes below.

Isaac Sim App Launcher will run as default when launching Isaac Sim from the Omniverse Launcher.

Isaac Sim Main App

This is the main windowed Isaac Sim application.

It is the first option and default mode to be launched from the Isaac Sim App Launcher.

To run Isaac Sim Main App:

  1. See Isaac Sim on Omniverse Launcher to run the Isaac Sim App Launcher.

  2. Click Launch

  3. Proceed to Running For The First Time

Isaac Sim Headless App (via WebSockets)

This is a command line version of Isaac Sim. It can be run remotely on a workstation with an RTX GPU and be accessed on a browser using the WebSockets protocol.

To run this mode:

  1. See Isaac Sim on Omniverse Launcher to run the Isaac Sim App Launcher.

  2. Select Isaac Sim (Headless WebSocket)

  3. Click Launch

  4. Proceed to Livestream WebSocket to live-stream Omniverse Isaac Sim remotely.

  5. Proceed to Running For The First Time

Isaac Sim Headless App (via Kit Remote)

This is a command line version of Isaac Sim. It can be run remotely on a workstation with an RTX GPU and be accessed from the Kit Remote client app available for Linux and Windows.

To run this mode:

  1. See Isaac Sim on Omniverse Launcher to run the Isaac Sim App Launcher.

  2. Select Isaac Sim (Headless Kit Remote)

  3. Click Launch

  4. Proceed to Connect to Omniverse Isaac Sim to live-stream Omniverse Isaac Sim remotely.

  5. Proceed to Running For The First Time

Isaac Sim Headless App (via WebRTC)

This is a command line version of Isaac Sim. It can be run remotely on a workstation with an RTX GPU and be accessed on a browser using the WebRTC protocol.

To run this mode:

  1. See Isaac Sim on Omniverse Launcher to run the Isaac Sim App Launcher.

  2. Select Isaac Sim (Headless WebRTC)

  3. Click Launch

  4. Proceed to Livestream WebRTC to live-stream Omniverse Isaac Sim remotely.

  5. Proceed to Running For The First Time

Isaac Sim Python

This is a mini app to run the Python samples.

Nucleus Workstation

To run the Omniverse Isaac Sim Samples, it is required to connect to a local or remote Nucleus server with the sample assets.

Cache Workstation

Omniverse Cache is useful when accessing files from some remote Nucleus servers by optimizing data transfers.

Adding Sample Assets

There are two methods to add the Omniverse Isaac Sim sample assets to the Nucleus Server.

Method #1: Package

The first method is to download the asset package and manually upload it to a Nucleus server. This method allows editing to the files and it does not require the Nucleus server to have access the Internet.

Steps

  1. Use the omniverse user credentials by setting the environment variables OMNI_USER and OMNI_PASS. See Setting default Username and Password connecting to Nucleus Server.

    Note

    If you are not using the omniverse user, you can set user permissions in the User Managemnt section by clicking the bottom left icon of the Omniverse Nucleus Web UI. Make sure the user is in the gm group.

  2. Run Omniverse Isaac Sim and connect to the desired Nucleus server in the Content tab.

  3. Right-click on the section to the right and select New Folder.

    ../_images/isaac_main_nucleus_create_folder1.png
  4. Create the Isaac folder.

    ../_images/isaac_main_nucleus_create_folder2.png
  5. Download the Isaac Sim Asset Package.

  6. Extract the package to a temporary folder.

  7. Login to the Nucleus server via the Web UI as the omniverse user. For localhost, go to http://localhost:8080/login. See Nucleus Web Service.

  8. Upload the extracted package to this Isaac folder.

    ../_images/isaac_main_nucleus_isaac_folder.png

Method #2: Mount

The second method is to create a mount on the Omniverse Nucleus server. This method requires the Nucleus Server to have access to the Internet and no storage is used on the Nucleus server. The mounted folder is also read-only.

Note

This mounted Isaac folder will be automatically updated with a new versions of Isaac Sim. The user does not need to update this folder

Steps

  1. Login to the Nucleus server via the Web UI as the omniverse user. For localhost, go to http://localhost:8080/login. See Nucleus Web Service.

  2. Create a New mount on the root folder with the settings below:

    Name: Isaac
    Type: Amazon S3
    Host: nvidia-omniverse-isaac.s3-us-west-1.amazonaws.com
    Service: s3
    Redirection: http://d30atda72sku6a.cloudfront.net
    
    ../_images/isaac_main_nucleus_add_mount.png

The folder should look like this:

../_images/isaac_main_nucleus_mount_folder.png

Docker & Cloud Deployment

Docker Container Deployment

Container Requirements

  1. See Omniverse Isaac Sim Container on NGC to get access to the Isaac Sim Container on NGC.

  2. Here are the requirements for running Omniverse Isaac Sim in a container:

  3. Here are the requirements for running Omniverse Nucleus:

  4. See Nucleus Cloud Deployment to install Omniverse Nucleus in the cloud.

  5. See Adding Sample Assets to Headless Nucleus to add the Omniverse Isaac Sim sample assets to Nucleus in the cloud.

  6. See Cache Installation to install Omniverse Cache in the cloud.

Isaac Sim Main App on Windowed Container

This section shows you how to run the Omniverse Isaac Sim container on local Ubuntu.

Steps

  1. See Container Requirements for container setup prerequisites.

  2. Run the command below to confirm your GPU driver version.

    $ nvidia-smi
    
  3. Using command line to login into NGC before downloading the Omniverse Isaac Sim container.

    $ sudo docker login nvcr.io
    Username: $oauthtoken
    Password: <Your NGC API Key>
    WARNING! Your password will be stored unencrypted in /home/username/.docker/config.json.
    Configure a credential helper to remove this warning. See
    https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    Login Succeeded
    
  4. Run the command below to enable xserver access:

    $ xhost +local:
    non-network local connections being added to access control list
    
  5. Run:

    $ sudo docker run --entrypoint ./runapp.sh --gpus all -e "ACCEPT_EULA=Y" --rm --network=host -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/localtime:/etc/localtime:ro -e DISPLAY=unix${DISPLAY} nvcr.io/nvidia/isaac-sim:2021.1.1
    

    This will download and launch the Omniverse Isaac Sim container in windowed mode.

    Note

  6. See Nucleus Cloud Deployment to install the Nucleus Server.

  7. See Adding Sample Assets to add the Omniverse Isaac Sim sample assets to a Nucleus Server.

  8. Proceed to Running For The First Time

Isaac Sim Headless Container

This section shows you how to run the Omniverse Isaac Sim container on remote Ubuntu workstation. Below are the steps to take while you are SSH into the remote workstation.

Steps

  1. See Container Requirements for container setup prerequisites.

  2. Run the command below to confirm your GPU driver version.

    $ nvidia-smi
    
  3. Using command line to login into NGC before downloading the Omniverse Isaac Sim container.

    $ sudo docker login nvcr.io
    Username: $oauthtoken
    Password: <Your NGC API Key>
    WARNING! Your password will be stored unencrypted in /home/username/.docker/config.json.
    Configure a credential helper to remove this warning. See
    https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    Login Succeeded
    
  4. Run:

    $ sudo docker run --entrypoint ./runheadless.websocket.sh --gpus all -e "ACCEPT_EULA=Y" --rm --network=host nvcr.io/nvidia/isaac-sim:2021.1.1
    

    This will download and launch the Omniverse Isaac Sim container in headless mode.

    Note

  5. Proceed to Livestream Clients to live-stream Omniverse Isaac Sim remotely.

Cloud Deployment

Here are the complete steps to run Omniverse Isaac Sim in the cloud.

Steps

  1. Setup a remote machine in the Cloud.

  2. Run Isaac Sim on the remote machine.

  3. Access Isaac Sim using a livestream client.

  4. Install Omniverse Nucleus on the same remote machine.

  5. Add the Omniverse Isaac Sim sample assets to Nucleus.

  6. Optionally install Omniverse Cache on the same remote machine.

  7. Run the Isaac Sim samples.

AWS Requirements

Here are the requirements for running Omniverse Isaac Sim on Amazon Web Services(AWS):

  1. An Amazon Web Services(AWS) account that is able to launch an EC2 instance with GPU support.

  2. Recommended specification for AWS EC2 Instance:

    • Instance Type: g4dn.4xlarge or higher

    • AMI: IsaacSim-Ubuntu-18.04-GPU-2021-05-25

To launch the AWS EC2 instance, please follow these steps below.

Steps

  1. Create your Key Pair PEM file by following the 8 steps in this Key Pair Guide.

    To prevent permission errors when trying to SSH into the AWS instance:

    On Linux, remember to chmod 400 yourkey.pem as instructed in the link above.

    On Windows, right click on “yourkey.pem” file, Properties:

    • Go to Security tab, click “Advanced”

    • Remove inheritance

    • Set current user as owner of the file and full permissions to only that user.

  2. From your AWS EC2 Console, click Launch Instance, select Community AMIs, search for “IsaacSim-Ubuntu-18.04-GPU-2021-05-25”, press Select.

    Note

    The AMI may not be visible to Hong Kong, Africa Cape Town, Middle East Bahrain and Milan zones.

    Select AMI
  3. Scroll down and pick a GPU instance (g4dn.4xlarge or higher). Click Next: Configure Instance Details at bottom right corner.

    ../_images/isaac_main_gpu_instance.png
  4. Make sure Auto-assign public IP is enabled.

    ../_images/isaac_main_auto_assign_ip.png
  5. Click Add Storage at the bottom right. Set the Root volume to size bigger than 8GB (like 32GB), else Docker won’t be able to extract the container from NGC due to insufficient storage space.

    ../_images/isaac_main_storage.png
  6. Click Next: Add Tags and click Next: Configure Security Group.

    Click Add Rule, select All traffic and My IP so the Livestream Clients and Visual Studio Code can see this instance’s IP address. If you are running a Livestream Client on another workstation, select the Custom source and enter the external IP address of that workstation.

    ../_images/isaac_main_all_traffic.png
  7. Click Review and Launch. Click Launch. Point to your existing PEM key or create a new PEM key (create a new one will save the key in your browser Download folder). Select Acknowledge.

    Click Launch Instance, it’ll give you a link, something similar to i-0edc9523b2fff2e44, click that instance link.

    ../_images/isaac_main_launch.png

    Note

    If you see “You have requested more vCPU capacity than your current vCPU limit”, you might need to stop your running instances to free up the vCPU usage and try again.

  8. Click Connect next to that Launch Instance button

    ../_images/isaac_main_connect_aws.png
  9. To SSH into the AWS instance:

    As shown above, on Linux or Windows, run this command from where your pem key folder is, replace the ubuntu@ec2-52-53-177-94.us-west-1.compute.amazonaws.com with your instance:

    $ ssh -i "yourkey.pem" ubuntu@ec2-52-53-177-94.us-west-1.compute.amazonaws.com
    

    On Windows, you can also use Putty to SSH:

    Convert your PEM key to Putty key.

    See Connecting to Your Linux Instance from Windows Using PuTTY

    Put your instance name above in the host name:

    ../_images/isaac_main_host_name_putty.png

    Select SSH/Auth, point Browse to your converted Putty key. Then click Open and Yes, it should be able to SSH in now.

    ../_images/isaac_main_auth.png

Azure Requirements

Here are the requirements for running Omniverse Isaac Sim on Microsoft Azure:

  1. A Microsoft Azure Portal account that is able to launch a Virtual Machine with GPU support.

  2. Recommended specification for an Azure Virtual Machine :

    • Size: NC4as_T4_v3

    • Image: Ubuntu Server 18.04 LTS - Gen 1

To launch the Azure Virtual Machine, please follow these steps below.

Steps

  1. Follow the steps in Create a Linux virtual machine in the Azure portal to create and connect to a VM with the specification above.

  2. See Isaac Sim Driver Requirements to install NVIDIA Drivers on the VM.

  3. See Isaac Sim Container Requirements to other dependencies on the VM.

Running Headless Container on the Cloud

To run Omniverse Isaac Sim on a remote machine or in the Cloud, please follow these steps in this section.

Steps

  1. Connect to the machine via SSH.

  2. Using command line to login into NGC before downloading the Omniverse Isaac Sim container.

    $ sudo docker login nvcr.io
    Username: $oauthtoken
    Password: <Your NGC API Key>
    WARNING! Your password will be stored unencrypted in /home/username/.docker/config.json.
    Configure a credential helper to remove this warning. See
    https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    Login Succeeded
    
  3. Run:

    $ sudo docker run --gpus all -e "ACCEPT_EULA=Y" --rm --network=host -v ~/docker/isaac-sim/documents:/root/Documents:rw -v ~/docker/isaac-sim/cache:/root/.cache/ov:rw -v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw -v ~/docker/isaac-sim/data:/root/.local/share/ov/data:rw nvcr.io/nvidia/isaac-sim:2021.1.1
    

    This will download and launch the Omniverse Isaac Sim container in headless (Websocket) mode by default.

    Note

  4. Proceed to Livestream Clients to live-stream Omniverse Isaac Sim remotely.

Nucleus Cloud Deployment

To run the Omniverse Isaac Sim Samples, it is required to connect to a Nucleus server with the sample assets. This section will guide you on installing the Nucleus server in the cloud and uploading the Omniverse Isaac Sim sample assets.

See Nucleus Overview for more information.

Nucleus Requirements

Here are the requirements for running Nucleus:

  1. Ubuntu 18.04 Operating System

Nucleus Cloud Installation

Steps

  1. Download the Nucleus Linux Installer

  2. To copy the Nucleus installer to the Cloud instance:

    Run:

    $ scp -r -i "<path_to_yourkey.pem>" "<path_to_downloaded_nucleus_installer>" ubuntu@<aws_instance_ip_address>:/home/ubuntu         #For AWS
    $ scp -r -i "<path_to_yourkey.pem>" "<path_to_downloaded_nucleus_installer>" azureuser@<azure_vm_ip_address>:/home/azureuser       #For Azure
    
  3. To SSH into the Cloud instance:

    Run:

    $ ssh -i "path_to_yourkey.pem" ubuntu@<aws_instance_ip_address>       #For AWS
    $ ssh -i "path_to_yourkey.pem" azureuser@<azure_vm_ip_address>        #For Azure
    
  4. Run:

    $ mkdir -p ~/.nvidia-omniverse/config
    $ mkdir -p ~/.config/autostart
    $ sudo apt install unzip -y
    $ unzip <nucleus_installer.zip> -d nucleus_installer
    $ cd nucleus_installer
    $ setup/nucleus-setup -i      # To install
    $ setup/nucleus-setup -u      # To uninstall
    
  5. To access the Nucleus server from a local instance of Omniverse Isaac Sim, use localhost:3009.

Note

If this Nucleus package is installed on a headless machine it can only be accessed locally to that machine.

A Nucleus Workstation in a desktop environment is recommended to enable sharing.

For enterprise users. See Nucleus Docker

Adding Sample Assets to Headless Nucleus

Steps

  1. Download the Isaac Sim Asset Package.

  2. To copy the Isaac Sim Asset Package to the Cloud instance:

    $ scp -r -i "<path_to_yourkey.pem>" "<path_to_downloaded_isaacsim_assets>" ubuntu@<aws_instance_ip_address>:/home/ubuntu         #For AWS
    $ scp -r -i "<path_to_yourkey.pem>" "<path_to_downloaded_isaacsim_assets>" azureuser@<azure_vm_ip_address>:/home/azureuser       #For Azure
    
  3. To SSH into the Cloud instance:

    $ ssh -i "path_to_yourkey.pem" ubuntu@<aws_instance_ip_address>       #For AWS
    $ ssh -i "path_to_yourkey.pem" azureuser@<azure_vm_ip_address>        #For Azure
    
  4. Run:

    $ sudo apt install unzip -y
    $ sudo mkdir -p docker/isaac-sim/documents/Isaac
    $ sudo unzip <isaac-sim-assets.zip> -d docker/isaac-sim/documents/Isaac
    $ sudo chmod -R 775 docker/isaac-sim/documents/Isaac
    
  5. Go to the Livestream Clients that is running Isaac Sim.

    ../_images/isaac_sample_assets_livestream.png
  6. Click on the Content tab and go to omniverse://localhost/

  7. Right-click on the right window and select Upload Files and Folders

    ../_images/isaac_sample_assets_upload1.png
  8. Go to /root/Documents and select the Isaac folder

    ../_images/isaac_sample_assets_upload2.png
  9. Click Open to start uploading.

    ../_images/isaac_sample_assets_upload3.png
  10. Proceed to Running For The First Time

Cache Installation

Cache is an optional app if you are connecting to external Omniverse Nucleus servers.

Steps

  1. Download the Cache Linux Installer.

  2. To copy the Cache installer to the Cloud instance:

    Run:

    $ scp -r -i "<path_to_yourkey.pem>" "<path_to_downloaded_cache_installer>" ubuntu@<aws_instance_ip_address>:/home/ubuntu         #For AWS
    $ scp -r -i "<path_to_yourkey.pem>" "<path_to_downloaded_cache_installer>" azureuser@<azure_vm_ip_address>:/home/azureuser       #For Azure
    
  3. To SSH into the Cloud instance:

    Run:

    $ ssh -i "path_to_yourkey.pem" ubuntu@<aws_instance_ip_address>       #For AWS
    $ ssh -i "path_to_yourkey.pem" azureuser@<azure_vm_ip_address>        #For Azure
    
  4. Run:

    $ sudo apt install unzip -y
    $ unzip <cache_installer.zip> -d cache_installer
    $ cd cache_installer
    $ setup/cache-setup -i      # To install
    $ setup/cache-setup -u      # To uninstall
    

Livestream Clients

This section shows you the three different methods of livestreaming Omniverse Isaac Sim.

Before running a livestream client, make sure that Isaac Sim app is loaded and ready. It may take a few minutes for Isaac Sim to be completed loaded.

To confirm this, look out for this line in the console or the logs:

Isaac Sim Headless Websocket App is loaded.

You may see other line depending on which mode was launched:

Isaac Sim Headless Kit Remote App is loaded.
Isaac Sim Headless WebRTC App is loaded.
Isaac Sim App is loaded.
Isaac Sim Python App is loaded.

Note

  • To cleanly exit the app. Click the File menu then click Exit.

Livestream WebSocket

This is a Livestream extension using the WebSocket protocol. The WebSocket client can be accessed using the browser at this URL:

http://<ip address>:8211/streaming/client

  • See Livestream WebSocket for more information.

  • Click on the red Play button to begin live streaming

Omniverse Kit Remote Client

This is a streaming client to view Omniverse Isaac Sim remotely on your desktop or workstation without a powerful GPU.

Kit Remote Client Requirements

Ubuntu 18.04

Download the Omniverse Kit Remote Linux Client

Please install these packages to run the streaming client:

$ sudo apt-get install libavcodec57 libavformat57 libavutil55 libsdl2-dev libsdl2-2.0-0

Windows 10

Connect to Omniverse Isaac Sim

Below are steps to live-stream a remote deployment of Omniverse Isaac Sim to you local Linux or Windows workstation using Omniverse Kit Remote Client:

  1. Get the IP address of your Omniverse Isaac Sim instance.

  2. Run Omniverse Kit Remote Client

    • On Linux:

      $ ./kit-remote.sh -s <remote_ip_address>
      
    • On Windows:

      > kit-remote.exe -s <remote_ip_address>
      

    Note

    • Replace <remote_ip_address> above with your Omniverse Isaac Sim instance Public IP.

    • To connect to a local headless instance of Omniverse Isaac Sim, use IP 127.0.0.1.

    • Each Omniverse Isaac Sim instance can only connect to one Omniverse Kit Remote Client. Connecting to an Omniverse Isaac Sim instance that is currently serving a Omniverse Kit Remote Client will result in an error for the second user.

    • The Omniverse Kit Remote Client will create a window of resolution 1280x720 by default. It is recommended to set your display resolution to 1080p(1920x1080). Use flag -w 1920 -h 1080 to set the Kit Remote windows resolution to 1080p.

    • Errors will still be displayed in the logs even when running Omniverse Kit Remote Client is successful. This will be cleaned up later on. Right now, the logs would be useful for debugging. See: Known Issues.

  3. Proceed to Running For The First Time

Livestream WebRTC

This is an experimental Livestream extension using the WebRTC protocol. It is similar to the Livestream WebSocket client using the browser but at a different URL:

http://<local_ip_address>:8211/streaming/webrtc-demo/?server=<local_ip_address>

Note

  • This livestream mode works locally but may have issues connecting remotely.

  • Click on the red Play button to begin live streaming

Debugging & Editing Python Code

You can edit Omniverse Isaac Sim Python code in real-time and view the changes. Any text editor can be used to edit the Python file but Visual Studio Code is preferred as it has an extension that allow accessing files on a remote workstation and easily modifying them.

Visual Studio Code Local Connection

Below are steps to access a local deployment of Omniverse Isaac Sim:

  1. Download and install Visual Studio Code.

  2. The samples source files are at:

exts/omni.isaac.samples/omni/isaac/samples/scripts
  1. You can edit the source files (leonardo_preview.py). Save and see the changes hot-loaded immediately. For example, change the label on the “Create Scenario” button to “Create My Awesome Scenario”:

self._create_franka_btn = ui.Button("Create My Awesome Scenario", clicked_fn=self._on_environment_setup)
  1. Save the code change with Ctrl + S and you should see the new button if you re-open the Isaac Examples -> Scenes -> Franka Block Stacking window.

    ../_images/isaac_main_hotloaded.png

Debugging with Visual Studio Code

Below are the steps for debugging python scripts in VS Code.

Launching from VS Code

  1. From the Isaac Sim App Launcher select the “Open in Terminal” button, then execute the following command: code . This launches a new VS Code window and opens the current folder. You can also launch VS Code and open the folder.

  2. To start debugging, open your python file and place breakpoints as needed. Select the “Run” icon from the toolbar on the left, and ensure “Current File” is selected from the configuration dropdown menu. Click “Start Debugging” or press F5 to launch the debugger. It’ll hit your breakpoints. Pressing F10 will step line by line. You can mouse over to examine variable values.

    ../_images/setup_vscode_debug.png
  3. If you need to pass arguments to your code, specify these in the “args” field of the .vscode/launch.json file. For example, running the generator.py sample directly from command line was done as follows:

    ./python.sh python_samples/syntheticdata/offline_generation/generator.py --scenario omniverse://<server-name>/Isaac/Samples/Synthetic_Data/Stage/warehouse_with_sensors.usd --num_frames 10 --max_queue_size 500
    

    Now if we want to run generator.py with breakpoints in VS Code, we specify those arguments in the launch.json:

    {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "env": {
                "EXP_PATH": "${workspaceFolder}/source/apps"
            },
            "envFile": "${workspaceFolder}/.vscode/.python_samples.env",
            "preLaunchTask": "setup_python_env",
            "args": ["--scenario=omniverse://<server-name>/Isaac/Samples/Synthetic_Data/Stage/warehouse_with_sensors.usd",
                        "--num_frames=10", "--max_queue_size=500"]
    }
    

    After modifying and saving the launch.json, press F5 to launch the debugger and F10 to step again.

  4. There are also isaac-sim [release] and [debug] configurations from the dropdown menu that will launch the Isaac Sim editor for debugging in VS Code.

Attaching the Debugger to a Running App

To debug a script that you are already running, use the VS Code Debugger extension.

  1. From the top toolbar select Window > Extensions. Then search for “vscode” and click the Enable button for the omni.kit.debug.vscode extension. By default, the status will show “VS Code Debugger Unattached”, in red text.

    ../_images/setup_vscode_extension.png
  2. Then launch VS Code, and select the “Run” icon from the toolbar on the left. From the configuration menu, select “Python: Attach (windows-x86_64/linux-x86_64) then press play. Notice that the status in Isaac Sim changes to “VS Code Debugger Attached” in blue text. You can now return to your python file in VS Code and add breakpoints to debug, as described above.

Visual Studio Code Remote Connection

Preparation Steps

Below are the steps needed on the remote workstation or Cloud instance where Omniverse Isaac Sim was deployed:

  1. To get the docker container ID, run:

    $ sudo docker ps
    
  2. Copy the sample folders from the running container to the host:

    $ sudo docker cp <container_id>:/isaac-sim/exts/omni.isaac.samples/omni/isaac/samples/ ~/docker/isaac-sim/samples
    $ sudo docker cp <container_id>:/isaac-sim/python_samples/ ~/docker/isaac-sim/python_samples
    $ sudo docker cp <container_id>:/isaac-sim/ros_samples/ ~/docker/isaac-sim/ros_samples
    $ sudo docker cp <container_id>:/isaac-sim/ros2_samples/ ~/docker/isaac-sim/ros2_samples
    
  3. Stop the docker container:

    $ sudo docker stop <container_id>
    
  4. Make the samples folder on host writable:

    $ sudo chmod -R 777 ~/docker/isaac-sim/samples
    $ sudo chmod -R 777 ~/docker/isaac-sim/python_samples
    $ sudo chmod -R 777 ~/docker/isaac-sim/ros_samples
    $ sudo chmod -R 777 ~/docker/isaac-sim/ros2_samples
    
  5. Re-run the container with:

    $ sudo docker run --gpus all --rm -e "ACCEPT_EULA=Y" -v ~/docker/isaac-sim/samples:/isaac-sim/exts/omni.isaac.samples/omni/isaac/samples -v ~/docker/isaac-sim/python_samples:/isaac-sim/python_samples -v ~/docker/isaac-sim/ros_samples:/isaac-sim/ros_samples -v ~/docker/isaac-sim/ros2_samples:/isaac-sim/ros2_samples -v ~/docker/isaac-sim/documents:/root/Documents:rw -v ~/docker/isaac-sim/cache:/root/.cache/ov:rw -v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw -v ~/docker/isaac-sim/data:/root/.local/share/ov/data:rw --network=host nvcr.io/nvidia/isaac-sim:2021.1.1
    

    Now, any changes to the ~/docker/isaac-sim/samples folder will be synchronized with the docker container.

    Note

Edit Python Code Remotely

Below are the steps needed on the local workstation which will be accessing the remote deployment of Omniverse Isaac Sim:

  1. Download and install Visual Studio Code.

  2. Install the Remote-SSH extension extension in Visual Studio Code.

  3. To SSH into the docker container, run SSH connection command (replace the <remote_ip_address> with Cloud instance public IP or the IP address to the remote workstation in the previous section):

    $ ssh -i "<path_to_yourkey.pem>" <username>@<remote_ip_address>
    
  4. Run Visual Studio Code with the Remote-SSH extension above. Then click the bottom left button ><.

  5. Select Remote-SSH: Open Configuration File ….

  6. Select the first file (C:\Users\<youruser>.ssh\config).

  7. Add the following and replace <public_ip_address> with your IPv4 Public IP above:

    Host Remote Isaac Sim
        HostName <remote_ip_address>
        IdentityFile <path_to_yourkey.pem>
        User <username>
    
    ../_images/isaac_main_vscode_connect.png
  8. Save the config file above.

  9. Click that bottom left button again \ Connect to host \ Remote Isaac Sim.

    You can open the Leonardo Preview source code folder (Ctrl + Shift + E or the 2 page button at top left cornerOpen Folder) at /var/lib/isaac-sim. Click OK.

    ../_images/isaac_main_open_folder.png
  10. The source files are at:

    ~/docker/isaac-sim/samples
    ~/docker/isaac-sim/python_samples
    ~/docker/isaac-sim/ros_samples
    ~/docker/isaac-sim/ros2_samples
    

    You can edit the source files (leonardo_preview.py). Save and see the changes hot-loaded immediately.

    For example, change the label on the “Create Scenario” button to “Create My Scenario”:

    self._create_franka_btn = self._window.layout.add_child(omni.kit.ui.Button("Create My Scenario"))
    

    Save the code change with Ctrl + S and you should see the new button is hot-loaded and appear immediately without any recompilation.

    ../_images/isaac_main_hotloaded.png
  11. To check the source changes are persistent:

    • Use another terminal to SSH into your remote workstation or Cloud instance.

    • Run sudo docker ps to get the docker container ID

    $ sudo docker ps
    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
    65083dc18bc3 nvcr.io/nvidia/isaac-sim:2021.1.1 "/bin/sh -c 'service…" About an hour ago Up About an hour 0.0.0.0:47995-48012->47995-48012/tcp, 0.0.0.0:47995-48012->47995-48012/udp, 0.0.0.0:49000-49007->49000-49007/tcp, 0.0.0.0:49000-49007->49000-49007/udp, 0.0.0.0:5000->22/tcp unruffled_bose
    
  12. To stop that docker, run this, replacing 65083dc18bc3 with your docker ID

    $ sudo docker stop 65083dc18bc3
    
  13. Run the docker command again and you should see your source changes (_create_another_franka_btn) still persist.