Installation#

Latest Container Release: 1.8.0

Once all the requirements are met, follow these steps to prepare the deployment system and deploy the containers into the Kubernetes cluster.

Prepare Deployment System#

To set up the deployment on your local machine or CI/CD tooling, make sure to install the following software packages with the supported versions.

kubectl: 1.29
Helm: 3.14.0

Add Helm Repositories#

To deploy the Streaming APIs, ensure your deployment system can access the repositories where the Helm charts are stored.

NVIDIA Helm Charts#

To register the NVIDIA-provided Helm charts with your local Helm setup, run the following command:

helm repo add omniverse https://helm.ngc.nvidia.com/nvidia/omniverse/ --username='$oauthtoken' --password=$NGC_API_TOKEN

A similar console output should be displayed:

"omniverse" has been added to your repositories

Flux Helm Charts#

The Flux Helm chart is used to install the fluxd container:

helm repo add fluxcd-community https://fluxcd-community.github.io/helm-charts

Prometheus Helm Charts#

To register the NVIDIA-provided Helm charts with your local Helm setup, run the following command:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

Repo Update#

To update all Helm repositories, run the following command:

helm repo update

(Optional) Download Helm Charts and Values files#

You may find it useful to download the Helm charts to examine what is being created by the Helm chart.

You can use the Helm fetch command to download the Helm charts into a directory of your choice.

The fetch command looks like this:

helm fetch https://helm.ngc.nvidia.com/nvidia/omniverse/charts/kit-appstreaming-session-$APP_VERSION.tgz --username='$oauthtoken' --password=$NGC_API_TOKEN

Helm Example Values#

You can preview the default parameters (values) of a Helm chart by running the helm show values command:

helm show values kit-appstream-session

The Omniverse App Streaming API NGC Collection includes example values files that are tailored for deployments on AWS.

Download the Samples and Resources and store them in a folder.

In the installation examples below, these resources are stored in a helm/<chart name> structure:

flux2
memcached
kit-appstreaming-applications
kit-appstreaming-rmcp
kit-appstreaming-rproxy
kit-appstreaming-manager
kit-appstreaming-aws-nlb

Prepare Kubernetes Resources#

Create the following Kubernetes resources to install the Streaming API solution.

Create Kubernetes Namespace#

The solution currently requires the creation of one new Kubernetes namespace to deploy various resources. For simplicity, this document assumes the namespace omni-streaming is created and used.

Create an omni-streaming namespace:

kubectl create namespace omni-streaming

A similar console output should be displayed:

kubectl create namespace omni-streaming

namespace/omni-streaming created

Create Image Registry Pull Secret#

To pull container images from NGC, a Docker registry secret is required. This secret must be referenced in the values file of the various NVIDIA-owned services. The example values files in the Helm charts have been configured to use a secret called regcred.

The following command creates this secret in the omni-streaming namespace that was created above. Replace $NGC_API_TOKEN with your token:

kubectl create secret -n omni-streaming docker-registry regcred \
    --docker-server=nvcr.io \
    --docker-username='$oauthtoken' \
    --docker-password=$NGC_API_TOKEN \
    --dry-run=client -o json | \
    kubectl apply -f -

A similar console output should be displayed:

kubectl create secret -n omni-streaming docker-registry regcred \
            --docker-server=nvcr.io \
            --docker-username='$oauthtoken' \
            --docker-password=$NGC_API_TOKEN \
            --dry-run=client -o json |
            kubectl apply -f -

secret/regcred created

Note

Creating secrets manually is not recommended and the recommended method is to use a combination of tools such as the Kubernetes External Secrets Operator, Hashicorp Vault, or AWS Secret Manager.

All Helm charts and configurations provided by this document support secrets injected or created using the methods mentioned above.

Accessing the Values File#

To successfully deploy the pods, a values file is required. This YAML file contains configuration elements for the corresponding Helm chart.

There are three sets of values files in YAML format that are used to deploy these Helm charts:

Bundled with the Helm charts. To access these values, use the helm show values command. Alternatively, you can download the Helm chart and inspect the values file directly. The provided values file is generalized with many default options and is typically not edited by the user, as it sets defaults.
Samples and Resources. These have the most common deployment options. They are available on NGC and include many common options, making the Helm charts immediate usable.
Created by you. These are values files you create to match your specific deployment needs.

Helm values files can be overlaid on each other, with the most recently referenced file overriding any previous ones.

Official documentation for Helm Value Files.

Install Memcached Service#

Memcached is a critical component for enabling fast startup of Kit streaming sessions. It caches shader information that would otherwise need to be compiled at the startup of each container.

Ensure the values.yaml file configuration matches the expected node label for the memcached service pod to run on the correct worker node or node group dedicated for caching.

Note

memcached must be installed within the same Kubernetes namespace as the stream Kit application containers.

Install with Helm:#

helm upgrade --install \
  -n omni-streaming --create-namespace \
  -f helm/memcached/values.yml \
  --repo https://charts.bitnami.com/bitnami \
  memcached-service-r3 memcached

A similar console output should be displayed:

$ helm upgrade --install \
  -n omni-streaming --create-namespace \
  -f values.yml \
  --repo https://charts.bitnami.com/bitnami \
  memcached-service-r3 memcached
Release "memcached-service-r3" does not exist. Installing it now.
NAME: memcached-service-r3
LAST DEPLOYED: Tue Apr 23 20:05:21 2024
NAMESPACE: omni-streaming
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: memcached
CHART VERSION: 7.0.3
APP VERSION: 1.6.26

** Please be patient while the chart is being deployed **

Install Flux Helm Controller and Flux Source Controller#

Components of the CNCF project Flux are used to manage the deployment of streaming applications. This approach makes the deployment and the management of Helm repositories and the resources more declarative and easier to manage.

Configuring the Helm repository:#

helm repo add fluxcd-community https://fluxcd-community.github.io/helm-charts
helm repo update

Deploying the flux Controllers:#

kubectl create namespace flux-operators

helm upgrade --install \
  --namespace flux-operators \
  -f helm/flux2/values.yaml \
fluxcd fluxcd-community/flux2

Install Omniverse Resource Management Control Plane#

The Resource Management Control Plane Service is used for manage the deployment of streaming sessions.

The Omniverse Kit App Streaming use Flux’s Helm and Source controllers. To instantiate a streaming application, a Helmrepositories.source.toolkit.fluxcd.io (details) resource needs to be created. This CRD will have been automatically deployed when the Flux operators were installed.

This resource informs Flux how to access the Helm repository in which the streaming application Helm chart is located.

Configuring the Service#

By default, the provided values.yaml file includes the necessary information to configure a Helm repository for NGC.

It requires a pre-configured secret named ngc-omni-user, which must contain a username and a password.

These are standard Kubernetes secrets that can be managed via an external secret manager or created manually.

The example below shows how to manually create the required secret; however, as noted above, creating secrets manually is not recommended and the recommended method is to use a combination of tools such as the Kubernetes External Secrets Operator, Hashicorp Vault, or AWS Secret Manager.

kubectl create secret -n omni-streaming generic ngc-omni-user \
            --from-literal=username='$oauthtoken' \
            --from-literal=password=$NGC_API_TOKEN \
            --dry-run=client -o json |
            kubectl apply -f -

To use a differently named secret, update the values file with the name of the secret you want to use.

Open and edit the example values.yaml file in helm/kit-appstreaming-rmcp
Replace the value of the secretRef with the name to be used

helm_repositories:
  - name: ngc-omniverse
    url: https://helm.ngc.nvidia.com/nvidia/omniverse/
    interval: 5m
    secretRef: ngc-omni-user
You can include additional Helm registries to the values file by adding additional entries.

Using CRDs to Create Additional Helm Repositories (optional)#

You can also create Helm repositories using standard Kubernetes manifests.

An example manifest can be found in the manifests folder within the Samples and Resources under manifests/helm-repositories/ngc-omniverse.yaml

There is also a manifest available for using an OCI-based registry. You need to update it with a URL that points at an OCI-based registry.

It can be found under: manifests/helm-repositories/oci-omniverse.yaml

For example, they can be deployed with kubectl:

kubectl apply -f manifests/helm-repositories/ngc-omniverse.yaml -n omni-streaming

Note

The above example deploys the ngc-omniverse named repository. If you have already configured it differently, there is a naming collision.

Deploying the Service#

Note

The code snippets below include a variable for $APP_VERSION. This variable can be edited manually using the latest release version number or set using an environment variable (e.g., export APP_VERSION=). The latest container versions are listed at the top of this page for quick reference.

Run the following installation/upgrade command to install the RMCP Service:

helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-rmcp/values.yaml --version $APP_VERSION \
 rmcp omniverse/kit-appstreaming-rmcp

A similar console output should be displayed:

$ helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-rmcp/values.yaml --version $APP_VERSION \
  rmcp omniverse/kit-appstreaming-rmcp
Release "rmcp" does not exist. Installing it now.
NAME: rmcp
LAST DEPLOYED: Thu Mar 14 18:06:33 2024
NAMESPACE: omni-streaming
STATUS: deployed
REVISION: 1

Installing the Streaming Session Manager#

The Streaming Session Manager is responsible for managing and creating streaming sessions.

It can manage available capacity based on the desired number of concurrent sessions. Currently, a 1-to-1 mapping of session to GPU within the cluster is recommended.

Update values.yaml#

Capacity Tuning#

Open and edit the example values.yaml file in helm/kit-appstreaming-manager.
Set the session_capacity:

serviceConfig:
   # -- Maximum amount of sessions to spawn. default of -1 means unlimited sessions
   session_capacity: 10 # Assuming there are 10 GPUs available for streaming

TLS Encrypted Streams (optional)#

Note

This is not required if the NLB service is used.

To use TLS-encrypted streams, a hostname is required to which a certificate can be attached. This document and the service assume the use of external-dns for provisioning the hostname. Click the following link for more information about Traffic Encryption. If a different approach is needed, there are ways to customize the resolution of the endpoints. This is referenced in the following document: Customizing Endpoint Resolution.

Open and edit the example values.yaml file in helm/kit-appstreaming-manager. Set enable_wss to true and specify the base_domain:

serviceConfig:
  backend_csp_args:
    enable_wss: true
    base_domain: stream.acme.org

The base_domain is used in conjunction with an external-dns annotation to create a unique hostname for each stream. For example, avdsbm.stream.acme.org.

Run the following installation/upgrade command to install the Streaming Session Manager service integration using the more generic values.yaml file:

helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-manager/values_generic.yaml --version $APP_VERSION \
  streaming omniverse/kit-appstreaming-manager

AWS Only - AWS NLB Manager (optional)#

For use with AWS and the AWS NLB, and specifically for the next step involving the deployment of the AWS NLB Manager (LBM), the following entries need to be set:

streaming.serviceConfig.backend_csp_cls: "nv.svc.streaming._csp.AWS"
streaming.serviceConfig.backend_csp_args.nlb_mgmt_svc: http://nlb/nlb

This ensures that, instead of dynamically creating a network load balancer, which can take several minutes, the service communicates with the AWS NLB Manager to fetch pre-configured target groups and listeners. These listeners have exclusive access to a single stream and are assigned and opened when a stream starts. When a stream is terminated, the target group and listener for that stream are returned to the pool and can be used for a new session.

Run the following installation/upgrade command to install the Streaming Session Manager without the LBM service integration, using the more generic values.yaml file:

helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-manager/values_generic.yaml --version $APP_VERSION \
  streaming omniverse/kit-appstreaming-manager

A similar console output should be displayed:

$ helm upgrade --install   --namespace omni-streaming   -f helm/kit-appstreaming-manager/values.yaml --version $APP_VERSION streaming  omniverse/kit-appstreaming-manager

Release "streaming" does not exist. Installing it now.

W0314 18:21:24.539864  283327 warnings.go:70] annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead
NAME: streaming
LAST DEPLOYED: Thu Mar 14 18:21:23 2024
NAMESPACE: omni-streaming
STATUS: deployed
REVISION: 1
TEST SUITE: None

Install Application and Profile Manager (AP)#

The Applications and Profile Manager manages the available applications and runtime profiles.

Open and edit the example values.yaml file in helm/kit-appstreaming-applications.
If you already have a containerized Kit application to stream, please refer to the “Deploying & Configuring Containerized Kit Applications” guide for details on how to register your application.
Save the file.

Run the following installation/upgrade command to install the Application and Profile Manager:

helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-applications/values.yaml --version $APP_VERSION \
  applications omniverse/kit-appstreaming-applications

Similar console output should be displayed:

$ helm upgrade --install   --namespace omni-streaming   -f helm/kit-appstreaming-applications/values.yaml --version $APP_VERSION applications  omniverse/kit-appstreaming-applications
Release "applications" does not exist. Installing it now.
W0314 18:23:29.161562  283447 warnings.go:70] annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead
NAME: applications
LAST DEPLOYED: Thu Mar 14 18:23:27 2024
NAMESPACE: omni-streaming
STATUS: deployed
REVISION: 1
TEST SUITE: None

Additional documentation on the various values can be found in the NGC Documentation.

Install AWS NLB Manager (optional)#

This step is only needed for deployment to AWS.

Note

This is not supported and considered a temporary fix until AWS extends the AWS Load Balancer Controllers functionality.

In order to avoid long start-up times of the Omniverse Kit App Streaming on AWS, at least one Amazon Network Load Balancer needs to be pre-provisioned. The LBM service uses these NLB to create warmed up pools for listeners and target groups. These target groups are then selected and assigned to a stream dynamically using an AWS target group binding resource.

By default, the LBM starts assigning ports starting for each port protocol (configurable in the settings). This range can be changed, but these are the TCP and UDP ports that would need to be opened. Two ports are used per stream.

For additional details on port requirements, reference the port/security information earlier in this document.

Prerequisites#

Provisioned NLB(s)
Provisioned IAM Role for LBM with attached IAM Policy

Update values.yaml#

In the provided values.yaml file at least the following elements need to be updated:

nlb.ingress.enabled: must be set to “true” if you want to be able to reach the service from outside of the cluster (e.g., for operational and/or debugging purposes).
nlb.serviceAccount.enabled: Must be set to “true”.
nlb.serviceAccount.name: The name of the K8S Service Account to create which will assume below IAM Role (IRSA).
nlb.serviceAccount.roleArn: The ARN of the AWS NLB Manager IAM Role to be assumed by the K8S Service Account.
nlb.serviceConfig.resource.dns.alias.tag.key: The TAG KEY on managed NLBs that will be used to determine the DNS name (TAG VALUE) associated with the NLB (only needed when TLS is enabled).
nlb.serviceConfig.resource.lookup.tag.key: The TAG KEY on managed NLBs that is used during lookup to configure the resources. Both the TAG KEY and VALUE need to match on the NLB for it to be managed by the LBM service.
nlb.serviceConfig.resource.lookup.tag.value: The TAG VALUE on managed NLBs that is used during lookup to configure the resources. Both the TAG KEY and VALUE need to match on the NLB for it to be managed by the LBM service.
nlb.serviceConfig.resource.stream.limit: The maximum number of streaming sessions to be configured on NLBs. Due to AWS service limits for the number of Listeners that can be attached to an NLB, this must be a value between 1 and 25.
nlb.serviceConfig.resource.ports.tcp.port_start: The starting TCP port to configure NLB Listeners and Target Groups on for streaming sessions. This value is incremented sequentially, up to a maximum of stream.limit. This value and the associated range of ports must not intersect the list of UDP ports.
nlb.serviceConfig.resource.ports.tcp.tls.enabled: Must be set to True to enable TLS termination at the NLB Listener. If enabled, you must set the ssl_policy and certificate_arn settings as well.
nlb.serviceConfig.resource.ports.tcp.tls.ssl_policy: The AWS SSL policy to use for the TLS listener. At the time of writing this document, if enabled, AWS recommends using ELBSecurityPolicy-TLS13-1-2-2021-06. This must be a value from the TLS 1.3 security policies list (see TLS Listeners for your Network Load Balancer for more information).
nlb.serviceConfig.resource.ports.tcp.tls.certificate_arn: The ARN of the ACM Certificate used on the TLS Listener. This certificate must match the DNS name associated with the NLB.
nlb.serviceConfig.resource.ports.udp.port_start: The starting UDP port to configure NLB Listeners and Target Groups on for streaming sessions. This value is incremented sequentially, up to a maximum of stream.limit. This value and the associated range of ports must not intersect the list of TCP ports.

Install with Helm#

helm upgrade --install \
  --namespace omni-streaming \
  -f helm/kit-appstreaming-aws-nlb/values.yaml --version $APP_VERSION \
  lbm omniverse/kit-appstreaming-aws-nlb

Similar Console Output should be Displayed#

helm upgrade --install --namespace omni-streaming -f helm/kit-appstreaming-aws-nlb/values.yaml lbm omniverse/kit-appstreaming-aws-nlb

Release "lbm" does not exist. Installing it now.
NAME: lbm
LAST DEPLOYED: Fri May 31 11:50:45 2024
NAMESPACE: omni-streaming
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Installing CHART "kit-appstreaming-aws-nlb". CHART VERSION: 0.2.0. APP VERSION: 0.2.0

** Please be patient while the chart is being deployed. **