Is this page helpful?

Installation#

Latest Container Release: 1.12.1

Important

Omniverse Streaming on NVCF replaces Omniverse Kit App Streaming (OKAS) as the Reference Architecture for streaming Omniverse Kit applications at scale.

Once all the requirements are met, follow these steps to prepare the deployment system and deploy the containers into the Kubernetes cluster.

Prepare Deployment System#

To set up the deployment on your local machine or CI/CD tooling, make sure to install the following software packages with the supported versions.

  • kubectl: 1.29

  • Helm: 3.14.0

Add Helm Repositories#

To deploy the Streaming APIs, ensure your deployment system can access the repositories where the Helm charts are stored.

NVIDIA Helm Charts#

To register the NVIDIA-provided Helm charts with your local Helm setup, run the following command:

helm repo add omniverse https://helm.ngc.nvidia.com/nvidia/omniverse/ --username='$oauthtoken' --password=$NGC_API_TOKEN

A similar console output should be displayed:

"omniverse" has been added to your repositories

Flux Helm Charts#

The Flux Helm chart is used to install the fluxd container:

helm repo add fluxcd-community https://fluxcd-community.github.io/helm-charts

Prometheus Helm Charts#

To register the NVIDIA-provided Helm charts with your local Helm setup, run the following command:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

Note

The Applications, Streaming Manager, and RMCP example values files set monitoring.enabled: true, which creates ServiceMonitor resources (the AWS NLB Manager example values leave monitoring disabled). This requires the Prometheus Operator custom resource definitions (monitoring.coreos.com/v1) to already exist in the cluster; install prometheus-community/kube-prometheus-stack first, or set monitoring.enabled: false in each chart’s values. The generated ServiceMonitor carries the label release: prometheus, which must match your Prometheus instance’s serviceMonitorSelector.

Repo Update#

To update all Helm repositories, run the following command:

helm repo update

(Optional) Download Helm Charts and Values files#

You may find it useful to download the Helm charts to examine what is being created by the Helm chart.

You can use the Helm fetch command to download the Helm charts into a directory of your choice.

The fetch command looks like this:

helm fetch https://helm.ngc.nvidia.com/nvidia/omniverse/charts/kit-appstreaming-session-$APP_VERSION.tgz --username='$oauthtoken' --password=$NGC_API_TOKEN

Helm Example Values#

You can preview the default parameters (values) of a Helm chart by running the helm show values command:

helm show values kit-appstreaming-session

The Omniverse Kit App Streaming NGC Collection includes example values files that are tailored for deployments on AWS.

Download the Samples and Resources and store them in a folder.

In the installation examples below, these resources are stored in a helm/<chart name> structure:

  • flux2

  • memcached

  • kit-appstreaming-applications

  • kit-appstreaming-rmcp

  • kit-appstreaming-manager

  • kit-appstreaming-aws-nlb

Prepare Kubernetes Resources#

Create the following Kubernetes resources to install the Streaming API solution.

Create Kubernetes Namespace#

The solution currently requires the creation of one new Kubernetes namespace to deploy various resources. For simplicity, this document assumes the namespace omni-streaming is created and used.

Create an omni-streaming namespace:

kubectl create namespace omni-streaming

A similar console output should be displayed:

kubectl create namespace omni-streaming

namespace/omni-streaming created

Note

Kit App Streaming can be deployed into any single namespace; the examples use omni-streaming. As of release 1.12.1, the session_namespace value defaults to the Helm release namespace and scopes both streaming session deployment and the Streaming Manager’s status and route lookups. If you deploy into a different namespace, create the Application, ApplicationVersion, and ApplicationProfile resources in that namespace and adjust any example manifests and values files that reference omni-streaming explicitly.

Create Image Registry Pull Secret#

To pull container images from NGC, a Docker registry secret is required. This secret must be referenced in the values file of the various NVIDIA-owned services. The example values files in the Helm charts have been configured to use a secret called regcred.

The following command creates this secret in the omni-streaming namespace that was created above. Replace $NGC_API_TOKEN with your token:

kubectl create secret -n omni-streaming docker-registry regcred \
    --docker-server=nvcr.io \
    --docker-username='$oauthtoken' \
    --docker-password=$NGC_API_TOKEN \
    --dry-run=client -o json | \
    kubectl apply -f -

A similar console output should be displayed:

kubectl create secret -n omni-streaming docker-registry regcred \
            --docker-server=nvcr.io \
            --docker-username='$oauthtoken' \
            --docker-password=$NGC_API_TOKEN \
            --dry-run=client -o json |
            kubectl apply -f -

secret/regcred created

Note

Creating secrets manually is not recommended and the recommended method is to use a combination of tools such as the Kubernetes External Secrets Operator, Hashicorp Vault, or AWS Secret Manager.

All Helm charts and configurations provided by this document support secrets injected or created using the methods mentioned above.

Accessing the Values File#

To successfully deploy the pods, a values file is required. This YAML file contains configuration elements for the corresponding Helm chart.

There are three sets of values files in YAML format that are used to deploy these Helm charts:

  1. Bundled with the Helm charts. To access these values, use the helm show values command. Alternatively, you can download the Helm chart and inspect the values file directly. The provided values file is generalized with many default options and is typically not edited by the user, as it sets defaults.

  2. Samples and Resources. These have the most common deployment options. They are available on NGC and include many common options, making the Helm charts immediate usable.

  3. Created by you. These are values files you create to match your specific deployment needs.

Helm values files can be overlaid on each other, with the most recently referenced file overriding any previous ones.

Official documentation for Helm Value Files.

Install Memcached Service#

Memcached is a critical component for enabling fast startup of Kit streaming sessions. It caches shader information that would otherwise need to be compiled at the startup of each container.

Ensure the values.yml file configuration matches the expected node label for the memcached service pod to run on the correct worker node or node group dedicated for caching.

Note

memcached must be installed within the same Kubernetes namespace as the stream Kit application containers.

Warning

The release name memcached-service-r3 must be used exactly as shown. The session Helm chart defaults its shader-cache endpoint to hsscdns://memcached-service-r3 (streamingKit.shader_cache.uri), and no shipped ApplicationProfile overrides the uri (a few AWS samples disable the cache entirely with shader_cache.enabled: false). With a different release name the lookup fails silently and every session falls back to compiling shaders from scratch; if you must rename the release, override streamingKit.shader_cache.uri in your ApplicationProfile chartValues. The provided values also set -I 1024m (max item size) because compiled shader entries exceed memcached’s 1 MB default; node sizing guidance is in the requirements page.

Install with Helm:#

helm upgrade --install \
  -n omni-streaming --create-namespace  \
  -f helm/memcached/values.yml \
  --set image.repository=bitnamilegacy/memcached \
  memcached-service-r3 oci://registry-1.docker.io/bitnamicharts/memcached

A similar console output should be displayed:

$ helm upgrade --install \
  -n omni-streaming --create-namespace \
  -f values.yml \
  --set image.repository=bitnamilegacy/memcached \
  memcached-service-r3 oci://registry-1.docker.io/bitnamicharts/memcached
Release "memcached-service-r3" does not exist. Installing it now.
NAME: memcached-service-r3
LAST DEPLOYED: Tue Apr 23 20:05:21 2024
NAMESPACE: omni-streaming
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: memcached
CHART VERSION: 7.0.3
APP VERSION: 1.6.26

** Please be patient while the chart is being deployed **

Install Flux Helm Controller and Flux Source Controller#

Components of the CNCF project Flux are used to manage the deployment of streaming applications. This approach makes the deployment and the management of Helm repositories and the resources more declarative and easier to manage.

Configuring the Helm repository:#

helm repo add fluxcd-community https://fluxcd-community.github.io/helm-charts
helm repo update

Deploying the flux Controllers:#

kubectl create namespace flux-operators

helm upgrade --install \
  --namespace flux-operators \
  -f helm/flux2/values.yaml \
  --version 2.14.1 \
fluxcd fluxcd-community/flux2

Note

The Flux chart version is pinned deliberately. The Resource Management Control Plane creates HelmRelease resources using the API version configured by rmcp.serviceConfig.helm_flux_version, which supports v2beta2 (default) and v2beta1 only; the Flux GA v2 API is not supported. Do not upgrade Flux past a release that still serves the beta APIs. The related values helm_release_namespace (namespace for session HelmRelease objects, defaults to the release namespace) and helm_flux_release_timeout (default 5m, raise it when session images are large) are set in the RMCP chart values.

Install Omniverse Resource Management Control Plane#

The Resource Management Control Plane Service is used for manage the deployment of streaming sessions.

The Omniverse Kit App Streaming use Flux’s Helm and Source controllers. To instantiate a streaming application, a Helmrepositories.source.toolkit.fluxcd.io (details) resource needs to be created. This CRD will have been automatically deployed when the Flux operators were installed.

This resource informs Flux how to access the Helm repository in which the streaming application Helm chart is located.

Configuring the Service#

The Helm repository for NGC is configured through the rmcp.helmRepositories value. The provided example values.yaml file does not include this block by default; add it as shown below, or apply the HelmRepository manifest described in the next section.

It requires a pre-configured secret named ngc-omni-user, which must contain a username and a password.

These are standard Kubernetes secrets that can be managed via an external secret manager or created manually.

The example below shows how to manually create the required secret; however, as noted above, creating secrets manually is not recommended and the recommended method is to use a combination of tools such as the Kubernetes External Secrets Operator, Hashicorp Vault, or AWS Secret Manager.

kubectl create secret -n omni-streaming generic ngc-omni-user \
            --from-literal=username='$oauthtoken' \
            --from-literal=password=$NGC_API_TOKEN \
            --dry-run=client -o json |
            kubectl apply -f -

To use a differently named secret, update the values file with the name of the secret you want to use.

  1. Open and edit the example values.yaml file in helm/kit-appstreaming-rmcp

  2. Replace the value of the secretRef with the name to be used

    rmcp:
  helmRepositories:
    - name: ngc-omniverse
      url: https://helm.ngc.nvidia.com/nvidia/omniverse/
      interval: 5m
      secretRef: ngc-omni-user

    You can include additional Helm registries to the values file by adding additional entries.

    Note

    The key is rmcp.helmRepositories (camelCase, nested under rmcp). Helm silently ignores unknown value paths, so a misspelled or top-level key produces no HelmRepository resources and every stream request fails to resolve its session chart. The repository name must be ngc-omniverse, because the shipped ApplicationVersion resources reference the session chart as ngc-omniverse/kit-appstreaming-session.

Using CRDs to Create Additional Helm Repositories (optional)#

You can also create Helm repositories using standard Kubernetes manifests.

An example manifest can be found in the manifests folder within the Samples and Resources under manifests/helm-repositories/ngc-omniverse.yaml

There is also a manifest available for using an OCI-based registry. You need to update it with a URL that points at an OCI-based registry.

It can be found under: manifests/helm-repositories/ngc-omniverse-oci.yaml

For example, they can be deployed with kubectl:

kubectl apply -f manifests/helm-repositories/ngc-omniverse.yaml -n omni-streaming

Note

The above example deploys the ngc-omniverse named repository. If you have already configured it differently, there is a naming collision.

Deploying the Service#

Note

The code snippets below include a variable for $APP_VERSION. This variable can be edited manually using the latest release version number or set using an environment variable (e.g., export APP_VERSION=). The latest container versions are listed at the top of this page for quick reference.

Run the following installation/upgrade command to install the RMCP Service:

helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-rmcp/values.yaml --version $APP_VERSION \
 rmcp omniverse/kit-appstreaming-rmcp

A similar console output should be displayed:

$ helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-rmcp/values.yaml --version $APP_VERSION \
  rmcp omniverse/kit-appstreaming-rmcp
Release "rmcp" does not exist. Installing it now.
NAME: rmcp
LAST DEPLOYED: Thu Mar 14 18:06:33 2024
NAMESPACE: omni-streaming
STATUS: deployed
REVISION: 1

Installing the Streaming Session Manager#

The Streaming Session Manager is responsible for managing and creating streaming sessions.

It can manage available capacity based on the desired number of concurrent sessions. Currently, a 1-to-1 mapping of session to GPU within the cluster is recommended.

Update values.yaml#

Capacity Tuning#

  1. Open and edit the example values.yaml file in helm/kit-appstreaming-manager.

  2. Set the session_capacity:

    serviceConfig:
   # -- Maximum amount of sessions to spawn. default of -1 means unlimited sessions
   session_capacity: 10 # Assuming there are 10 GPUs available for streaming
Backend Service URLs#

The manager requires the URLs of the RMCP and Applications resolver services. The chart defaults are empty, so these must be set; the provided example values files already contain the working in-cluster values:

serviceConfig:
  backend_rmcp_url: "http://rmcp/rmcp"
  backend_applications_resolver_url: "http://resolver"

Note

The /rmcp path suffix must match the prefix_url set in the RMCP chart values (the example RMCP values set prefix_url: "/rmcp"). Similarly, the AWS nlb_mgmt_svc URL http://nlb/nlb depends on the NLB Manager’s root_path: "/nlb". If you clear either value in that service’s chart, drop the corresponding suffix here.

TLS Encrypted Streams (optional)#

Note

This is not required if the NLB service is used.

To use TLS-encrypted streams, a hostname is required to which a certificate can be attached. This document and the service assume the use of external-dns for provisioning the hostname. Click the following link for more information about Traffic Encryption. If a different approach is needed, there are ways to customize the resolution of the endpoints. This is referenced in the following document: Customizing Endpoint Resolution.

  1. Open and edit the example values.yaml file in helm/kit-appstreaming-manager. Set enable_wss to true and specify the base_domain:

    serviceConfig:
  backend_csp_args:
    enable_wss: true
    base_domain: stream.acme.org

The base_domain is used in conjunction with an external-dns annotation to create a unique hostname for each stream. For example, avdsbm.stream.acme.org.

Run the following installation/upgrade command to install the Streaming Session Manager service integration using the more generic values.yaml file:

helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-manager/values_generic.yaml --version $APP_VERSION \
  streaming omniverse/kit-appstreaming-manager
AWS Only - AWS NLB Manager (optional)#

For use with AWS and the AWS NLB, and specifically for the next step involving the deployment of the AWS NLB Manager (LBM), the following entries need to be set:

streaming.serviceConfig.backend_csp_cls: "nv.svc.streaming._csp.AWS"
streaming.serviceConfig.backend_csp_args.nlb_mgmt_svc: http://nlb/nlb

This ensures that, instead of dynamically creating a network load balancer, which can take several minutes, the service communicates with the AWS NLB Manager to fetch pre-configured target groups and listeners. These listeners have exclusive access to a single stream and are assigned and opened when a stream starts. When a stream is terminated, the target group and listener for that stream are returned to the pool and can be used for a new session.

Run the following installation/upgrade command to install the Streaming Session Manager without the LBM service integration, using the more generic values.yaml file:

helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-manager/values_generic.yaml --version $APP_VERSION \
  streaming omniverse/kit-appstreaming-manager

A similar console output should be displayed:

$ helm upgrade --install   --namespace omni-streaming   -f helm/kit-appstreaming-manager/values.yaml --version $APP_VERSION streaming  omniverse/kit-appstreaming-manager

Release "streaming" does not exist. Installing it now.

W0314 18:21:24.539864  283327 warnings.go:70] annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead
NAME: streaming
LAST DEPLOYED: Thu Mar 14 18:21:23 2024
NAMESPACE: omni-streaming
STATUS: deployed
REVISION: 1
TEST SUITE: None

Install Application and Profile Manager (AP)#

The Applications and Profile Manager manages the available applications and runtime profiles.

  1. Open and edit the example values.yaml file in helm/kit-appstreaming-applications.

  2. If you already have a containerized Kit application to stream, please refer to the “Deploying & Configuring Containerized Kit Applications” guide for details on how to register your application.

  3. Save the file.

Run the following installation/upgrade command to install the Application and Profile Manager:

helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-applications/values.yaml --version $APP_VERSION \
  applications omniverse/kit-appstreaming-applications

Similar console output should be displayed:

$ helm upgrade --install   --namespace omni-streaming   -f helm/kit-appstreaming-applications/values.yaml --version $APP_VERSION applications  omniverse/kit-appstreaming-applications
Release "applications" does not exist. Installing it now.
W0314 18:23:29.161562  283447 warnings.go:70] annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead
NAME: applications
LAST DEPLOYED: Thu Mar 14 18:23:27 2024
NAMESPACE: omni-streaming
STATUS: deployed
REVISION: 1
TEST SUITE: None

Additional documentation on the various values can be found in the NGC Documentation.

Install AWS NLB Manager (optional)#

This step is only needed for deployment to AWS.

Note

This is not supported and considered a temporary fix until AWS extends the AWS Load Balancer Controllers functionality.

In order to avoid long start-up times of the Omniverse Kit App Streaming on AWS, at least one Amazon Network Load Balancer needs to be pre-provisioned. The LBM service uses these NLB to create warmed up pools for listeners and target groups. These target groups are then selected and assigned to a stream dynamically using an AWS target group binding resource.

By default, the LBM starts assigning ports starting for each port protocol (configurable in the settings). This range can be changed, but these are the TCP and UDP ports that would need to be opened. Two ports are used per stream.

For additional details on port requirements, reference the port/security information earlier in this document.

Prerequisites#

  • Provisioned NLB(s)

  • Provisioned IAM Role for LBM with attached IAM Policy

Update values.yaml#

In the provided values.yaml file at least the following elements need to be updated:

  • nlb.ingress.enabled: must be set to “true” if you want to be able to reach the service from outside of the cluster (e.g., for operational and/or debugging purposes).

  • nlb.serviceAccount.enabled: Must be set to “true”.

  • nlb.serviceAccount.name: The name of the Kubernetes Service Account to create which will assume below IAM Role (IRSA).

  • nlb.serviceAccount.roleArn: The ARN of the AWS NLB Manager IAM Role to be assumed by the Kubernetes Service Account.

  • nlb.serviceConfig.resource.dns.alias.tag.key: The TAG KEY on managed NLBs that will be used to determine the DNS name (TAG VALUE) associated with the NLB (only needed when TLS is enabled).

  • nlb.serviceConfig.resource.lookup.tag.key: The TAG KEY on managed NLBs that is used during lookup to configure the resources. Both the TAG KEY and VALUE need to match on the NLB for it to be managed by the LBM service.

  • nlb.serviceConfig.resource.lookup.tag.value: The TAG VALUE on managed NLBs that is used during lookup to configure the resources. Both the TAG KEY and VALUE need to match on the NLB for it to be managed by the LBM service.

  • nlb.serviceConfig.stream.limit: The maximum number of streaming sessions to be configured on NLBs. Due to AWS service limits for the number of Listeners that can be attached to an NLB, this must be a value between 1 and 25.

  • nlb.serviceConfig.ports.tcp.port_start: The starting TCP port to configure NLB Listeners and Target Groups on for streaming sessions. This value is incremented sequentially, up to a maximum of stream.limit. This value and the associated range of ports must not intersect the list of UDP ports.

  • nlb.serviceConfig.ports.tcp.tls.enabled: Must be set to True to enable TLS termination at the NLB Listener. If enabled, you must set the ssl_policy and certificate_arn settings as well.

  • nlb.serviceConfig.ports.tcp.tls.ssl_policy: The AWS SSL policy to use for the TLS listener. At the time of writing this document, if enabled, AWS recommends using ELBSecurityPolicy-TLS13-1-2-2021-06. This must be a value from the TLS 1.3 security policies list (see TLS Listeners for your Network Load Balancer for more information).

  • nlb.serviceConfig.ports.tcp.tls.certificate_arn: The ARN of the ACM Certificate used on the TLS Listener. This certificate must match the DNS name associated with the NLB.

  • nlb.serviceConfig.ports.udp.port_start: The starting UDP port to configure NLB Listeners and Target Groups on for streaming sessions. This value is incremented sequentially, up to a maximum of stream.limit. This value and the associated range of ports must not intersect the list of TCP ports.

Note

Only the DNS alias tag key and the lookup tag key and value live under nlb.serviceConfig.resource; the stream limit and port settings sit directly under nlb.serviceConfig. Verify the rendered ConfigMap after install if streams use unexpected ports.

Install with Helm#

helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-aws-nlb/values.yaml --version $APP_VERSION \
  lbm omniverse/kit-appstreaming-aws-nlb

Similar Console Output should be Displayed#

helm upgrade --install --namespace omni-streaming -f helm/kit-appstreaming-aws-nlb/values.yaml lbm omniverse/kit-appstreaming-aws-nlb

Release "lbm" does not exist. Installing it now.
NAME: lbm
LAST DEPLOYED: Fri May 31 11:50:45 2024
NAMESPACE: omni-streaming
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Installing CHART "kit-appstreaming-aws-nlb". CHART VERSION: 1.12.1. APP VERSION: 1.12.1

** Please be patient while the chart is being deployed. **