Is this page helpful?

Shared Shader Caching#

Shader caching is a performance optimization technique used in GPU rendering to store the results of shader compilation for reuse in future sessions. Shaders are programs that run on the GPU to handle graphical tasks like lighting, shading, and other visual effects. These shaders must be compiled from source code into executable instructions before they can be run on the GPU, and this process can be time-consuming. Shader caching addresses this issue by storing the compiled shaders, significantly reducing the need for recompilation and improving performance.

Why Shader Caching is Needed#

The shader compilation process can introduce significant delays in applications that rely on GPU rendering, such as gaming, virtual environments, or 3D modeling. When shaders are not cached, they must be compiled every time the application runs, which leads to longer startup times and latency, especially in complex or resource-heavy scenes. By caching compiled shaders, applications can avoid repeated compilation, allowing for faster startup and a smoother user experience.

This is particularly important in distributed or containerized environments, where multiple sessions may be run on different GPU nodes. Without shader caching, every node might need to recompile the same shaders, leading to unnecessary delays and inefficient resource use. A shader cache minimizes these issues by storing compiled shaders for reuse across different nodes or sessions.

Shader Caching in Omniverse RTX Renderer#

Before the Omniverse RTX Renderer can render a USD scene, all shaders must be compiled by the GPU driver running on the GPU Worker Node. This process can introduce significant latency from the moment an Omniverse Kit application is launched until the user begins receiving the WebRTC stream.

Local Environments: In a local environment, the NVIDIA GPU driver caches the compiled shaders in a directory. The first time a shader is used, it incurs the cost of compilation, but on subsequent uses, the shader is pulled from the cache, significantly reducing startup time.

Containerized Environments: In containerized environments, however, the default behavior is different. Compiled shaders are written to the container’s virtual filesystem, which is not persistent across sessions. This means the compiled shaders are lost once the container is terminated. Moreover, subsequent invocations might run on different GPU worker nodes, which further complicates shader reuse. As a result, users may experience repeated shader compilation delays, leading to inefficient performance, especially when switching between dynamically allocated GPU nodes.

Solution: Shared Shader Cache#

To address the challenges in distributed environments, a shared shader cache can be implemented. This approach leverages NVIDIA technology to store compiled shaders in a key-value store, such as memcached, which is a fast, distributed, in-memory caching system. This setup allows multiple GPU nodes to share access to the same compiled shaders, avoiding redundant compilation and significantly reducing the time it takes for users to start streaming.

Kit Version Compatibility#

Note

Kit applications built on Kit 110.0 do not consume the shared shader cache, even when it is configured. Support is restored in Kit 110.1.2 and later: in Kit App Streaming 1.12.1, the usd-viewer container (110.1.3, built on Kit SDK 110.1.3) supports the shared shader cache. Kit applications built on Kit 109.x are unaffected.

How to Set Up a Shared Shader Cache#

To deploy a shared shader cache, perform the following steps:

  1. Start a memcached instance in the cluster. memcached serves as the key-value store for caching compiled shaders, enabling fast retrieval across different nodes. The memcached Service must be resolvable by the name in the cache URI from the namespace the streaming sessions run in; the installation guide deploys it in the same namespace under the expected name. Raise the memcached maximum item size; compiled shader entries exceed the 1 MB default (the provided deployment values set -I 1024m).

  2. Enable the shader cache for streaming sessions. The session Helm chart injects the AUTO_ENABLE_DRIVER_SHADER_CACHE_WRAPPER environment variable into the Kit application container when the streamingKit.shader_cache value is enabled:

    streamingKit:
  shader_cache:
    enabled: true
    uri: "hsscdns://<memcached service name>"

    Set this in the session chart values or override it per profile under spec.chartValues.streamingKit.shader_cache in an ApplicationProfile. The hsscdns:// scheme resolves the memcached instance by DNS name.

Once configured, the application pulls compiled shaders from the shared cache, speeding up the start of streaming and minimizing compilation delays across sessions. The first session after the cache is empty populates it and can take longer to reach readiness than an uncached session; subsequent sessions start faster.

Verifying the Cache is Working#

A configured cache can appear healthy while receiving no traffic, for example when the Kit application version does not consume it. Verify end to end with the memcached statistics counters:

kubectl exec <memcached pod> -n <namespace> -- sh -c 'echo stats | nc -w2 127.0.0.1 11211' | grep -E "cmd_set |get_hits |curr_items "

  • After the first streaming session loads a scene, cmd_set and curr_items increase as compiled shaders are written to the cache.

  • After a later session loads the same scene from a new pod, get_hits increases as shaders are read back, and the session reaches readiness faster.

  • If the counters stay at zero while sessions stream, the application is not consuming the cache. Check the Kit version compatibility note above and the environment variable configuration.

By implementing a shared shader cache, you can significantly reduce the start-up latency in distributed or containerized environments, improving the overall performance of GPU-accelerated applications.