Nucleus on Google Cloud#

Introduction#

This guide focuses on installing and configuring an Omniverse Enterprise Nucleus Server on Google Cloud. The requirements for a successful deployment are outlined along with the technical steps below.

Depending on requirements and/or internal policies, there may be some settings and/or options that are set differently than what is covered within this documentation. This document focuses on using the Cloud Shell launched from within the Google Cloud console, however standing up individual Virtual Machines manually using the web-based interface is also supported.

Solution Overview#

The following steps outline a reference solution that implements the primary components of a Nucleus deployment. To handle communication from end users, a Global Cloud Load Balancer is configured to direct and manage traffic to the Nucleus server. The Nucleus server runs on a Google Cloud instance with no external IP Address and only accepts traffic from the load balancer. To communicate with the NVIDIA Container Registry, a Cloud NAT is configured so the Nucleus server can access the Internet.

Attention

Steps within this document require network configuration changes to allow remote connectivity. Please consult your IT/Information Security team for proper advisement.

Prerequisites#

A Google Cloud account to deploy the necessary Virtual Machines to run an Enterprise Nucleus Server.
This tutorial uses gcloud and gsutil commands, which can run from a Cloud Shell instance launched from the Console. If it’s preferred to use gcloud and gsutil from a local workstation, install the Google Cloud CLI. The documentation explains how to run commands using the in-browser Cloud Shell; if it’s preferred to use the gcloud CLI from a workstation, adjust the instructions accordingly.
In the Google Cloud console, on the project selector page, select an existing or create a Google Cloud project.
Ensure a billing account is enabled for your new Google Cloud project. Click here to learn how to check if billing is enabled for your project.
Enable the Compute Engine API.
A Fully Qualified Domain Name (FQDN) that you plan to use for your Nucleus environment (e.g., nucleus.mydomain.com).
The ability to perform DNS changes (Creation of an A record.)

Attention

When copying the BASH and YAML code from this document to your workstation, use a text editor such as Notepad++ (Windows) or BBEdit (macOS) that supports Unix EOL (End of Line) conversion so the file is saved correctly. Some text editors add additional non-visible line breaks that may cause parsing issues.

Variable Legend#

The following variables are used throughout this document:

Variable	Description
SERVER_NAME	The name for the Nucleus server
GROUP_NAME	The name for the instance group
ARTIFACT_NAME	The file name of the Nucleus artifact
ZONE	The desired zone for the server
SIZE	The size of the boot disk in GB
PROJECT	The name of the project
ADDRESS_NAME	The name for the IP Address resource, not the IP Address itself
CERTIFICATE_NAME	The name of the certificate
DOMAIN	The chosen FQDN for the Nucleus server
REGION	The zone where you created the Nucleus server
SOURCE	The path to the YAML file
PROJECT_ID	The Project ID (not the Project Name) of your project

Note

It’s very helpful to make a list of the values for the variables above in a text file on your workstation as they will be used often within this documentation.

Provisioning your Environment#

Preparing your Cloud Shell environment#

Using Cloud Shell run the following commands to prepare your environment.

Set your project:

gcloud config set project [PROJECT_ID]
In the Console, you can locate your Project ID in the IAM & Admin Settings.

Ensure you have a default VPC network. If your project does not already have a default VPC network, create one:

gcloud compute networks create default \
  --subnet-mode=auto

Creating the Nucleus server instance#

The Nucleus server instance is created from a base Ubuntu Linux 22.04 LTS image. This example uses an e2-standard-8 (8 vCPU’s, 32GB RAM) machine type. Choose the sizing that best meets your performance requirements.

Using Cloud Shell, create the instance:

gcloud compute instances create [SERVER_NAME] \
  --zone=[ZONE] \
  --machine-type=e2-standard-8 \
  --network-interface=subnet=default,no-address \
  --tags=nucleus-server,allow-iap \
  --maintenance-policy=MIGRATE \
  --provisioning-model=STANDARD \
  --scopes=https://www.googleapis.com/auth/cloud-platform \
  --create-disk=\
auto-delete=yes,\
boot=yes,\
image-project=ubuntu-os-cloud,\
image-family=ubuntu-2204-lts,\
mode=rw,\
size=[SIZE],\
type=projects/[PROJECT]/zones/[ZONE]/diskTypes/pd-ssd \
  --shielded-secure-boot \
  --shielded-vtpm \
  --shielded-integrity-monitoring \
  --reservation-affinity=any

Note

When choosing the size of the instance, consider both the current and expected future growth.

Creating an unmanaged instance group#

In order for the Nucleus server to serve as the backend to a load balancer, it must be part of an instance group. The following details creating of the instance group and adding the Nucleus server to it.

Using Cloud Shell, create the instance group:

gcloud compute instance-groups unmanaged create [GROUP_NAME] --zone=[ZONE]

Add port mapping to the instance group to facilitate Nucleus server traffic:

gcloud compute instance-groups unmanaged set-named-ports \
[GROUP_NAME] \
  --zone=[ZONE] \
  --named-ports=\
api:3019,\
auth:3100,\
auth-login:3180,\
discovery:3333,\
lft:3030,\
search:3400,\
tagging:3020,\
web:80

Add the instance to the instance group:

gcloud compute instance-groups unmanaged add-instances [GROUP_NAME] \
  --instances=[SERVER_NAME] --zone=[ZONE]

Creating a static IP Address#

Create a static external IP Address for the load balancer. This IP Address will also be used to map the server to your domain.

Using Cloud Shell, create a static external IP Address:

gcloud compute addresses create [ADDRESS_NAME] \
  --global \
  --ip-version IPV4 \
  --network-tier=PREMIUM

Take note of the resulting address in the Console for the next step.

Using a DNS service such as domains.google.com, map the reserved external IP Address you created to a fully qualified domain name (FQDN) using an A record. (e.g., nucleus.mydomain.com)

Creating a certificate#

Create a certificate to provide a secure TLS/SSL connection to the Nucleus server.

Using Cloud Shell, create the certificate:

gcloud beta compute ssl-certificates create [CERTIFICATE_NAME] \
  --global \
  --domains=[DOMAIN]

Creating a firewall rules#

In this example, you create the nucleus-ports firewall rule. This is an ingress rule that allows traffic from the Google Cloud load balancer to your Nucleus server over specified ports. Google Cloud load balancers always use IP ranges, 130.211.0.0/22 and 35.191.0.0/16, so there is no need to change these values. This example uses the target tag nucleus-server to identify the VM.

Using Cloud Shell, create a firewall rule:

gcloud compute firewall-rules create nucleus-ports \
  --direction=INGRESS \
  --priority=1000 \
  --network=default \
  --action=ALLOW \
  --rules=\
tcp:80,tcp:3009,tcp:3010,tcp:3019,tcp:3020,\
tcp:3030,tcp:3100,tcp:3180,tcp:3333,tcp:3400 \
  --source-ranges=130.211.0.0/22,35.191.0.0/16 \
  --target-tags=nucleus-server

Create a health check and specify TCP Port 3333 to use for it:

gcloud compute health-checks create http http-basic-check \
  --request-path="/healthcheck" \
  --port 3333

As the Nucleus server has no external IP Address, connect to it using Identity-Aware Proxy (IAP). Create a firewall rule to allow IAP traffic to your Nucleus server:
gcloud compute firewall-rules create allow-ssh-from-iap \ --source-ranges 35.235.240.0/20 \ --target-tags allow-iap \ --allow tcp:22

Creating a Cloud NAT#

Your Nucleus server will not have an external IP Address, so cannot reach the Internet without a Cloud NAT and Cloud Router.

Create a Cloud Router:

gcloud compute routers create nat-router \
  --network default \
  --region [REGION]

Add a configuration to the router:

gcloud compute routers nats create nat-config \
  --router-region [REGION] \
  --router nat-router \
  --nat-all-subnet-ip-ranges \
  --auto-allocate-nat-external-ips

Creating the HTTPS load balancer#

You create an HTTPS Load Balancer to serve as a front end for the Nucleus server. An HTTPS Load Balancer provides a single endpoint to connect to the Nucleus server, while at the same time providing added security and traffic management.

Creating the backend services#

Omniverse Nucleus requires different traffic types to be routed in different ways. There are eight different backend services to create.

Copy this BASH script to a text file on your local workstation, edit it using a text editor of your choice, and ensure that the file ends with the extension of .sh. Then upload the file into your Cloud Shell environment.

#! /bin/bash

GROUP_NAME=[GROUP_NAME]
ZONE=[ZONE]
BACKENDS="api auth auth-login discovery lft search tagging web"

# Iterate over all backends.
for BACKEND in $BACKENDS; do

  # Create backend service.
  gcloud compute backend-services create nucleus-$BACKEND \
    --load-balancing-scheme=EXTERNAL_MANAGED \
    --protocol=HTTP \
    --port-name=$BACKEND \
    --health-checks=http-basic-check \
    --enable-cdn \
    --enable-logging \
    --timeout=30 \
    --global

  # Add backend service to instance group.
  gcloud compute backend-services add-backend nucleus-$BACKEND \
    --instance-group=$GROUP_NAME \
    --instance-group-zone=$ZONE \
    --balancing-mode "UTILIZATION" \
    --capacity-scaler=1 \
    --max-utilization=1 \
    --global

done

Wait until all backend services are created before proceeding to the next step.

Creating the URL map#

This step creates what is referred to as the load balancer resource itself. Using Cloud Shell, create a URL map to route the incoming requests to the default backend service:

gcloud compute url-maps create nucleus-lb \
  --default-service nucleus-web

Creating the HTTPS proxy#

The proxy is the portion of the load balancer that holds the TLS/SSL certificate for an HTTPS load balancer. You will also load your certificate in this step.

Using Cloud Shell, create a target HTTPS proxy to route requests to the load balancer:

gcloud compute target-https-proxies create nucleus-https-lb-proxy \
  --url-map=nucleus-lb \
  --ssl-certificates=[CERTIFICATE_NAME]

Create a global forwarding rule to route incoming requests to the proxy:

gcloud compute forwarding-rules create nucleus-https-443 \
  --load-balancing-scheme=EXTERNAL_MANAGED \
  --network-tier=PREMIUM \
  --address=[ADDRESS_NAME] \
  --global \
  --target-https-proxy=nucleus-https-lb-proxy \
  --ports=443

Create a global TLS/SSL policy and attach it to the HTTPS proxy:

gcloud compute ssl-policies create nucleus-ssl-policy \
  --profile MODERN \
  --min-tls-version 1.0

Attach the TLS/SSL policy to the global target HTTPS proxy:

gcloud compute target-https-proxies update nucleus-https-lb-proxy \
  --ssl-policy=nucleus-ssl-policy

Importing the URL map configuration file#

In Google Cloud, a URL map routes incoming web requests to backend services. You configure the mapping using YAML syntax. Copy this YAML code to a text file on your local workstation, edit it using the editor of your choice, and ensure that the file ends with the extension of .yaml. Then upload the file into your Cloud Shell environment.

Replace PROJECT with the Project ID of your Google Cloud project and DOMAIN with the FQDN of your chosen domain (e.g., nucleus.mydomain.com).

defaultUrlRedirect:
  hostRedirect: [DOMAIN]
  httpsRedirect: true
  prefixRedirect: /omni/web3/
  redirectResponseCode: FOUND
hostRules:
- hosts:
  - [DOMAIN]
  pathMatcher: path-matcher
kind: compute#urlMap
name: nucleus-lb
pathMatchers:
- defaultService: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-web
  name: path-matcher
  pathRules:
  - paths:
    - /
    urlRedirect:
      httpsRedirect: false
      pathRedirect: /omni/web3/
      stripQuery: false
  - paths:
    - /omni/web3/*
    routeAction:
      urlRewrite:
        pathPrefixRewrite: /
    service: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-web
  - paths:
    - /omni/api
    service: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-api
  - paths:
    - /omni/auth
    service: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-auth
  - paths:
    - /omni/auth/login/*
    routeAction:
      urlRewrite:
        pathPrefixRewrite: /
    service: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-auth-login
  - paths:
    - /omni/discovery/healthcheck
    routeAction:
      urlRewrite:
        pathPrefixRewrite: /healthcheck
    service: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-discovery
  - paths:
    - /omni/discovery/*
    service: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-discovery
  - paths:
    - /omni/discovery
    service: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-discovery
  - paths:
    - /omni/lft/*
    routeAction:
      urlRewrite:
        pathPrefixRewrite: /
    service: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-lft
  - paths:
    - /omni/search3
    service: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-search
  - paths:
    - /omni/tagging3
    service: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-tagging
  - paths:
    - /omniverse://*
    routeAction:
      urlRewrite:
        pathPrefixRewrite: /
    service: https://www.googleapis.com/compute/v1/projects/[PROJECT]/global/backendServices/nucleus-web

Import the URL map configuration file to the load balancer:

gcloud compute url-maps import nucleus-lb \
  --source=[SOURCE] \
  --global

Creating the HTTP load balancer#

In Google Cloud, you create a second HTTP load balancer to forward requests to the HTTPS load balancer. Copy this YAML code to a text file on your local workstation, edit it using the editor of your choice, and ensure that the file ends with the extension of .yaml. Then upload the file into your Cloud Shell environment.
kind: compute#urlMap name: nucleus-lb-redirect defaultUrlRedirect: redirectResponseCode: MOVED_PERMANENTLY_DEFAULT httpsRedirect: True

Using Cloud Shell, import the URL map configuration file to the load balancer:

gcloud compute url-maps import nucleus-lb-redirect \
  --source=[SOURCE] \
  --global

Create a target HTTP proxy to route requests to the load balancer, using the URL map you created:

gcloud compute target-http-proxies create nucleus-http-proxy \
  --url-map=nucleus-lb-redirect