AWS NLB Manager (optional)#

Using the AWS NLB Manager(LBM) is optional and only required if you want to leverage NLB pooling. This enables you to provision a set number of NLBs and dynamically bind/unbind listeners and target groups when a Kit application stream begins or ends.

Why use NLB Pooling?

By default, the Omniverse application streaming stack uses Kubernetes services of type Load Balancer to handle TCP (signaling) and UDP (media) traffic for the WebRTC stream.

Depending on the deployment environment, the time required to configure a load balancer and optionally a DNS entry through external-dns can vary significantly.

On AWS it can take up to several minutes, whereas other environments require only a few seconds.

To ensure equally fast startup times on AWS, you pre-provision a pool of NLBs for your cluster, eliminating the cost of dynamically creating the NLBs and their DNS entries. You then labeling them using a configurable tag.

The NLB Manager uses the configurable tag to identify the available pool of NLBs. It then adds target groups and listeners to them and dynamically configures the routing of application streams on instantiation and termination.

When using the LBM, the service type should be set to ClusterIP. TargetGroupBindings should then be used to route traffic from the load balancer to the streaming pod.

NLB Provisioning#

Pre-provision Network Load Balancers with the following configuration:

  • Internet-facing Load Balancer

  • NLB deployed in all public subnets and across all AZ used by the Kubernetes’ Worker Nodes

  • Cross-Zone Load Balancing enabled

  • Security Group configured (see below for details)

  • Add a tag key/value pair to identify the NLB as available for streaming

    • This key/value pair is given to the NLB service to filter NLBs

  • If requiring TLS:

    • Attach a certificate to the NLB

    • Configure a Route53 entry for the NLB and add a tag to the NLB:

    Route53Alias: <FQDN>

Tag Lookup Configuration#

The LBM supports dynamic configuration of NLBs at service startup and through the GET:/refresh API endpoint. These settings can be configured via the service’s application.toml and Helm chart values file using the following parameters:

nv.ov.svc.streaming.aws.nlb.resource.lookup.tag.key = ""
nv.ov.svc.streaming.aws.nlb.resource.lookup.tag.value = ""

Any NLBs with a matching tag key/value are configured by the LBM.

TLS Configuration#

The LBM supports configuring the signaling (TCP) port with TLS termination at the NLB listener. TLS requires both a valid ACM certificate and a DNS alias record pointing to the NLB’s public IP and/or default DNS name. The LBM searches for a configurable tag and use its value as the NLB’s alias for the streaming session.

This feature is enabled and configurable through the service’s application.toml and Helm chart values file using the following settings:

nv.ov.svc.streaming.aws.nlb.resource.dns.alias.tag.key = "Route53Alias"
nv.ov.svc.streaming.aws.nlb.ports.tcp.tls.enabled = true
nv.ov.svc.streaming.aws.nlb.ports.tcp.tls.ssl_policy = "ELBSecurityPolicy-TLS13-1-2-2021-06"
nv.ov.svc.streaming.aws.nlb.ports.tcp.tls.certificate_arn = "<arn>"

Port and Security Group Configuration#

The LBM manages the creation of Listener(s) and Target Group(s) for the provided Network Load Balancers. By default, the following ports are used:

NLB Port Allocation

Default value

Signaling

TCP 443

Media

UDP 80

Note

The LBM’s TCP and UDP starting port and stream limit (per NLB) are configurable through the service’s application.toml and Helm chart values file using the following settings:

nv.ov.svc.streaming.aws.nlb.stream.limit = 1
nv.ov.svc.streaming.aws.nlb.ports.tcp.port_start = 443
nv.ov.svc.streaming.aws.nlb.ports.udp.port_start = 80

At startup, the LBM attempts to pre-populate the necessary listeners and target groups based on the provided configuration.

In the above example (default configuration), two listeners (one TCP and one UDP) and target groups are created on ports 443 and 80; therefore, traffic to these ports must be allowed via security groups attached to the following resources:

  • The Security Group(s) of the Network Load Balancer(s) themselves, allowing traffic from:

    • The client source address(es)

    • The NAT gateway EGRESS IP address

  • The Security Group(s) of the EKS (GPU) Node(s) themselves, allowing traffic from:

    • The Network Load Balancer(s)

    • The client source address(es)

IAM Role & Policy#

The LBM requires an IAM role, which NVIDIA recommends have its own Kubernetes service account. The IAM role and policy closely follow the IAM setup of the AWS Load Balancer Controller using IRSA. However, unlike the role and policy used for the AWS Load Balancer Controller, the IAM policy for the LBM should be scoped to the minimal resources needed.

For detailed setup instructions, please refer to the official documentation above. The following details and links are provided as a quick reference only.

IAM Role Trust#

The configured IAM Roles need to have a Trust Policy created for the Kubernetes service account of the AWS NLB Manager to assume the role via the IAM OIDC Identity Provider.

For more information:

IAM Policy Permissions Example#

The IAM policy for the AWS NLB Manager role must include the following permissions:

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": [
            "elasticloadbalancing:ModifyListener",
            "tag:GetResources",
            "elasticloadbalancing:DescribeTags",
            "elasticloadbalancing:CreateTargetGroup",
            "elasticloadbalancing:RemoveListenerCertificates",
            "elasticloadbalancing:DescribeLoadBalancerAttributes",
            "elasticloadbalancing:DescribeLoadBalancers",
            "elasticloadbalancing:CreateListener",
            "elasticloadbalancing:DescribeTargetGroupAttributes",
            "elasticloadbalancing:DescribeListeners",
            "elasticloadbalancing:ModifyRule",
            "elasticloadbalancing:AddTags",
            "elasticloadbalancing:CreateRule",
            "elasticloadbalancing:DescribeTargetHealth",
            "elasticloadbalancing:DescribeTargetGroups",
            "elasticloadbalancing:DescribeListenerCertificates",
            "elasticloadbalancing:AddListenerCertificates",
            "elasticloadbalancing:DescribeRules",
            "elasticloadbalancing:ModifyTargetGroup"
         ],
         "Resource": "*"
      }
   ]
}
On this page