Cloud XR

NVIDIA Omniverse™ CloudXR is an AR/VR extension bringing power and ease to your Omniverse Projects. In just a few clicks, Omniverse can stream your USD content to any supported iOS or Android device and deliver AR/VR walk-around capability.

Prerequisites

NVIDIA Omniverse™ CloudXR is expected to be run on iOS and Android Devices and therefore will require both an AR capable iOS or Android Device with Android Studio.

Mandatory

Before getting started, the following utilities are necessary.

  • NVIDIA Omniverse™ Kit version 2021.3 or greater

  • For Android devices:

    • Android Studio v3.4+ or Android Command Line Tools.

    • Android Tablet with AR capabilities. (need a list of supported devices)

Optional

For expert Android developers looking for a more complete environment for debugging, the following is suggested.

  • A fully setup Android development environment

    • Android SDK

    • Android NDK and Android Platform Tools

    • OpenJDK 1.8

NVIDIA Omniverse™ Kit setup

Set Firewall Settings in Windows

Omniverse CloudXR requires bi-directional communication between the Connected Device and your computer. Because of this, firewall settings need to be adjusted.

  • Press the Windows Key and type “Settings”, Click on the Settings icon when it appears in the start menu.

  • Select Network and Internet

  • Select Windows Firewall

  • Select Advanced Settings

  • Select Inbound Rules

  • Locate any disabled Omniverse KIT Rules

  • Right Click on each of the rules and Select Properties

  • Select Allow the Connection

  • Once all rules have been allowed, Select Outbound Rules and repeat the process

If No Inbound Rules exist:

  • Click on New Rule > Program > This Program Path

  • Browse to the location of your Omniverse Kit/Create/View installation. Depending on your installation method, this could be one of:

    • %LocalAppData%\ov\pkgData\ + Release Version + \kit\kit.exe or

    • C:\Program Files\NVIDIA Corporation\Omniverse\Kit\ + Release Version + \_build\windows-x86_64\release\kit.exe

  • Select Allow the Connection then select Domain , Private and Public

  • Name it Omniverse Kit

  • Click on Finish

Install the Kit AR Tablet Extension

  • Launch NVIDIA Omniverse™ Kit

  • Click Window -> Extensions

  • Search for “cloudxr”

  • Select the “Kit AR Tablet” and click “Install”

  • Installing this requires re-starting the application to take effect.

Start NVIDIA Omniverse™ Kit with NVIDIA Omniverse™ CloudXR

  • Navigate to Omniverse Kit/Create/View application

  • Create Shortcut with these CloudXR flags

    • Navigate to your Kit installation (see above)

    • Right click on kit.exe and select Create Shortcut

    • You will be prompted to create the shortcut on the desktop. Select Accept.

    • Navigate to your desktop and locate the shortcut.

    • Right Click on the Shortcut and select Properties

    • At the end of the target line, ADD --enable omni.kit.xr.profile.tabletar example: "C:\Users\USERNAME\AppData\Local\ov\pkg\create-2021.3.2\kit.exe" --enable omni.kit.xr.profile.tabletar

  • Launch Omniverse Kit/Create/View using Shortcut

    • Double Click your new Shortcut

Kit Tablet AR Settings

Kit Tablet AR settings are typically found on the bottom right panel if NVIDIA Omniverse™ Kit is running the Kit AR Tablet extension.

  • Change Selected Output Plugin to CloudXR

  • Set Scale Override to ON

  • Set Predefined Scale to Scale of the Asset Disable

Render Settings for AR Mode

In order for the augmented reality (AR) mode to function correctly with the remote device’s camera, the following graphics options need to be set correctly.

  • In Render Settings -> Post Processing:

    • XR Compositing: must be checked ON

Connecting with the iOS Client

  • Install the iOS app named Omniverse Remote from the App Store or TestFlight.

  • Launch Omniverse Remote

  • Enter your NVIDIA Omniverse™ Kit server’s IP address in the Server Address field.

  • You can toggle some settings for AR/VR and 30/60 FPS modes here if desired.

  • Tap Connect

    • In Camera View (AR) mode, you will need to move your iOS device around so that it can find the ground plane. Once you see a grid appear on the ground, tap anywhere in that grid to begin streaming your scene.

    • You can use the directional pad buttons to move around in your scene, or simply walk around and change your iPad’s camera view.

    • When finished, tap the X icon in the top-right of your screen

Android Device setup

Android SDK

The Android tools needed by Omniverse can be attained by the more robust Android Studio or by the Command Line Tools.

Install and Configure Android Studio

The main reason NVIDIA Omniverse™ CloudXR needs android studio is to get the APK tools and to configure our environment.

  • Download Android Studio

  • Once downloaded Install

  • Once installed launch Android Studio

  • Upon starting Android Studio, Select “Configure > Configure SDK” to adjust the SDK properties

  • Select the SDK Platform Tab

    • Check-On Android 11

    • Check-On any additional Android Versions to match your target Android Device(s)

  • Select the SDK Tools Tab

    • Check-On Google USB Driver

  • Select OK/Apply to continue and install the necessary components.

  • Once installed, you can close Android Studio

Create Environment Variables

Once we have Android Studio installed we need to create Environment Variables for the Platform Tools

  • In Windows, Open a File Explorer window and Select the address Bar

  • Type %localappdata% to open your local user app data folder.

  • Open Android, then SDK, then platform-tools

  • In the address bar and select and copy the path

  • Press the windows Start Button amd type “Env”

  • Select “Edit the System Environment Variables” to open the System Properties panel on the Advanced Tab

  • Select “Environment Variables”

  • In the System Variables list, Locate Path and select “Edit”

  • In the resulting window press “New”

  • Paste the path to the Platform Tools folder and add a trailing \

    • Typical Path C:\Users\USERNAME\AppData\Local\Android\Sdk\platform-tools\

  • Press OK to commit the system variable

  • Test adb

    • Open a command prompt and type adb help

    • If the help output is delivered from adb, you have successfully installed the android environment and created the necessary paths.

Connecting Your Android Device

  • Enable Developer mode on the target device

  • Enable Debug Mode on the target device

  • Connect the target device to the host workstation

  • Open a command prompt and type adb devices

    • If connected, your device should be listed

Install NVIDIA Omniverse™ CloudXR on Your Device

  • Navigate to NVIDIA Omniverse™ CloudXR Installer

    • If installed with defaults, path will be C:\Program Files\NVIDIA Corporation\Omniverse\Kit\ + Installed Kit Version Number + _build\windows-x86_64\release\xrclients\googlear

    • Copy the path from the Windows Explorer Address Bar

  • Launch Command Prompt in Administrator Mode

    • Press Start Key on Keyboard and type cmd

    • Select “Run As Administrator” on the Command Prompt

  • Install NVIDIA Omniverse™ CloudXR

    • Type cd + <space> then Paste (Ctrl+V) the path and hit enter.

    • Check that device is still connected and not “locked” or “asleep” then type adb install ar-sample.apk

  • If the install process hangs on “Error: Device not authorized”

    • Press Ctrl + C to stop adb from executing

    • On the android device, look for authorization notification

    • Accept/Authorize the host computer and repeat Install CloudXR step

  • On your android device a new application called CloudXR ARCore Client should be available in your application list.

Connecting with the Android Client

  • Launch CloudXR ARCore Client on your Android Device.

  • Enter the IP Address of Host Computer on CloudXR Server IP Address line in NVIDIA Omniverse™ CloudXR Settings.

  • Use Device to slowly scan your surroundings. Allow device to create a “Grid”.

  • Tap Screen on any grid location to select where you want NVIDIA Omniverse™ CloudXR to Display your Omniverse Kit Content

  • The mesh should now appear and allow you to review your model in AR on your device.

Note

Once your model appears, you can reposition your model to another place on your grid by a long tap and hold on the screen. When the grid shows up again tap the new position to re-position the model.