Streaming Omniverse to Apple Vision Pro and iPad: CloudXRKit Client Overview#

Note

Applies to: Spatial Extensions, Kit 109.0.3+, CloudXR 6

The Apple client path uses CloudXRKit, a native Swift framework, to stream spatial XR content from an Omniverse Kit server to Apple Vision Pro and iPad. The reference client application is the cloudxr-apple-generic-viewer.

Supported Devices#

visionOS 2.4+

Apple Vision Pro

  • Immersive headset (6DoF)

  • Native hand tracking

  • Spatial audio

  • Eye tracking

iPadOS 18.4+

iPad Pro

  • Tablet form factor

  • Touch interaction

  • ARKit integration

Architecture#

flowchart LR KitServer["Kit XR Server\nCloudXR 6 (Native)"] -- "CloudXR Native Protocol\nStereo Video + Audio\nPose Data + Data Channel" --> GenericViewer["Generic Viewer App\n(CloudXRKit)"] GenericViewer -- "Head / Hand Pose\nOpaque Data Messages" --> KitServer

The Apple client connects directly to the Kit server’s CloudXR 6 (Native) runtime. The connection carries:

Server to Client

  • Rendered stereo frames

  • Spatial audio

  • Opaque data channel messages

Client to Server

  • Head pose

  • Hand tracking data

  • Opaque data channel messages

Key Features#

6DoF Head Tracking

Full positional and rotational tracking on Vision Pro.

Hand Tracking

Native visionOS hand tracking with optional visualization (finger joint cubes).

Opaque Data Channels

Bidirectional messaging through the MessageChannel API for application-level communication (JSON payloads).

Audio Streaming

Spatial audio from the server (Windows servers only).

SwiftUI Interface

Native UI for configuration, connection, and in-session controls.

Resolution Presets

Configurable streaming resolution for bandwidth/quality tradeoff.

GDN Support

Connect through the NVIDIA Graphics Delivery Network (GDN) for cloud deployments. GDN hosts the Kit application on NVIDIA-managed GPU infrastructure and streams to the client over the internet, eliminating the need for a local server workstation. In the generic viewer, select a GDN zone instead of entering a manual IP address. Network requirements for GDN differ from local streaming — a minimum of 100 Mbps downstream and 20 Mbps upstream is recommended.

Generic Viewer App Structure#

The generic viewer provides a complete reference implementation:

Key Source Files#

Component

Description

ViewerApp.swift

App entry point, registers CloudXRKit systems

AppModel.swift

Session state management

SessionConfigView.swift

Connection configuration UI (IP, zone, auth, resolution)

ServerActionsView.swift

Opaque data channel testing UI (Action buttons)

ImmersiveView.swift

visionOS immersive rendering using RealityKit

StreamingView+iOS.swift

iOS/iPadOS streaming rendering

Kit Server Requirements#

The Kit server must be configured with:

  1. XR extensions enabled (refer to Create a Kit XR App)

  2. OpenXR Runtime set to CloudXR 6 (Native)

  3. XR started – click Start XR in the XR panel

Important

Use CloudXR 6 (Native) for Apple clients. The CloudXR 6 (WebRTC) runtime is for Meta/web clients and will not work with CloudXRKit.

Next Steps#

Setup and Build

Clone, configure Xcode, and build the app for Vision Pro or iPad.

Apple Client: Setup and Build
Connect to Server

Configure and connect to your Kit XR server.

Apple Client: Connect to Server
Code Snippets

Ready-to-use Swift snippets for gestures, messaging, portals, and more.

CloudXR Cookbook: Common Feature Snippets