Omniverse Spatial Core Concepts: XRCore, Profiles, Coordinate Systems, and Frame Scheduling#

Note

Applies to: Spatial Extensions, Kit 109.0.3+, CloudXR 6

This core concepts reference covers the fundamental concepts and architecture of the Spatial Extensions system.

XR System Architecture#

The Kit XR system is built on a modular architecture that separates concerns and enables flexibility:

graph TB App[Kit Application] --> XRCore[XRCore Singleton] XRCore --> Profiles[Profile System] Profiles --> VR[VR Profile] Profiles --> AR[AR Profile] Profiles --> Tablet[TabletAR Profile] XRCore --> Systems[System Plugins] Systems --> OpenXR[OpenXR Plugin] Systems --> SimXR[SimulatedXR Plugin] XRCore --> Input[Input Management] Input --> Controllers[Controllers] Input --> HMD[HMD/Head] Input --> Eyes[Eye Tracking] XRCore --> USD[USD Integration] USD --> XRUsdLayer[XRUsdLayer] XRUsdLayer --> Stage[USD Stage] XRCore --> Events[Event System] Events --> MessageBus[Message Bus] XRCore --> Settings[XR Settings]

XRCore Singleton#

The XRCore is the main entry point for all XR functionality. It is implemented as a singleton that manages the entire XR system lifecycle.

Accessing XRCore#

import omni.kit.xr.core

# Get the singleton instance
xr_core = omni.kit.xr.core.XRCore.get_singleton()

Key Responsibilities#

Profile Management: Enable/disable XR experiences
Input Coordination: Manage controllers and devices
Frame Scheduling: Coordinate rendering pipeline
Event Distribution: Message bus for XR events
USD Integration: Create and manage XRUsdLayers

Profile System#

Profiles configure the XR experience for different use cases. Each profile defines render settings, input configurations, and behavior patterns.

Available Profiles#

VR

Use Case: Virtual Reality experiences with full immersion

Characteristics:

Stereoscopic rendering (one image per eye)
Opaque blend mode
Controller-based interaction
Foveated rendering support
Quality presets (performance/balanced/quality)

Enable:

XRCore.request_enable_profile("vr")

AR

Use Case: Augmented Reality with real-world overlay

Characteristics:

Multiple alpha-blend modes: alpha (passthrough composite), additive (holographic), opaque (fully virtual)
Real-world pass-through
Controller interaction optional
Matte object support for shadow casting onto real surfaces
Optimized for see-through displays

Enable:

XRCore.request_enable_profile("ar")

TabletAR

Use Case: Tablet-based AR experiences

Characteristics:

Tablet form factor optimization
Touch-based navigation
Fixed viewport orientation
CloudXR streaming support

Enable:

XRCore.request_enable_profile("tabletar")

Profile Lifecycle#

sequenceDiagram participant App participant XRCore participant Profile participant System participant Renderer App->>XRCore: request_enable_profile("vr") XRCore->>Profile: Load VR profile settings Profile->>System: Initialize XR system (OpenXR) System->>Renderer: Configure stereo rendering Renderer-->>XRCore: Ready XRCore-->>App: xr.enable event XRCore->>App: xr_display.enable event Note over App,Renderer: XR Experience Running App->>XRCore: request_disable_profile() XRCore->>Renderer: Stop rendering XRCore->>System: Shutdown XR system XRCore-->>App: xr.disable event

Checking Profile Status#

# Check if XR is enabled
if xr_core.is_xr_enabled():
    print("XR system is active")

# Check if display is enabled
if xr_core.is_xr_display_enabled():
    print("XR display is rendering")

# Get current profile
current_profile = xr_core.get_current_profile()
profile_name = current_profile.get_name()
print(f"Active profile: {profile_name}")

# List all available profiles
profiles = xr_core.get_profile_name_list()
print(f"Available profiles: {profiles}")

System Plugins#

System plugins handle communication with XR runtimes. The plugin architecture allows supporting multiple XR platforms.

OpenXR Plugin#

Extension: omni.kit.xr.system.openxr

Implements the OpenXR standard for cross-platform VR/AR support.

Supported Runtimes:

SteamVR
Windows Mixed Reality
Oculus Runtime
CloudXR (for streaming)
Meta Quest (through Link or Air Link)
Varjo
HTC Vive

Configuration:

import carb.settings
settings = carb.settings.get_settings()

# Use system OpenXR runtime
settings.set("/xr/persistent/system/openxr/runtime", "system")

# Or use CloudXR
settings.set("/xr/persistent/system/openxr/runtime", "cloudxr")

# Select OpenXR as system
settings.set("/persistent/xr/profile/vr/system/display", "OpenXR")

SimulatedXR Plugin#

Extension: omni.kit.xr.system.simulatedxr

Simulates an XR system for testing without hardware.

Use Cases:

Development without HMD
Automated testing
Performance benchmarking
CI/CD pipelines

Configuration:

settings.set("/persistent/xr/profile/vr/system/display", "SimulatedXR")
settings.set("/xr/simulatedxr/enabled", True)

# Configure simulated refresh rate
settings.set("/xr/profile/vr/simulatedxr/refreshRate", 90.0)

Frame Scheduling#

Understanding frame scheduling is crucial for optimal performance in XR.

The Two-Frame Pipeline#

Kit XR uses a 2-frame latency pipeline to balance performance and latency:

graph LR Frame_N[Frame N] Frame_N1[Frame N+1] Frame_N2[Frame N+2] subgraph "Simulation Thread" Sim_N[Simulate N] Sim_N1[Simulate N+1] Sim_N2[Simulate N+2] end subgraph "Render Thread" Render_N[Schedule N] Render_N1[Schedule N+1] GPU_N[GPU Render N] end Sim_N --> Render_N Sim_N1 --> Render_N1 Render_N --> GPU_N

Frame Lifecycle#

Frame N:

Simulation: USD updates, physics, animations
USD-RT Merge: Combine USD and USD-RT changes
Pose Capture: Read controller/HMD poses (late as possible)
Pre-Sync Events: Python callbacks in sync with rendering
USD-RT Update: Update managed objects (beams, arcs, controllers)
Schedule Rendering: Queue frame for GPU

Frame N+1 (Parallel): 7. Post-Sync Events: Handle button presses, business logic 8. Next Frame Prep: Prepare for next simulation cycle

Scheduling Events#

XRCore emits events at specific points in the frame:

import carb.events

message_bus = xr_core.get_message_bus()

# Pre-sync: Executes in sync with rendering
pre_sync_sub = message_bus.create_subscription_to_pop_by_type(
    carb.events.type_from_string("xr.pre_sync_update"),
    lambda e: print("Pre-sync - about to render")
)

# Post-sync: Executes after rendering scheduled (1 frame delay)
post_sync_sub = message_bus.create_subscription_to_pop_by_type(
    carb.events.type_from_string("xr.post_sync_update"),
    lambda e: print("Post-sync - handle business logic")
)

When to Use:

Pre-Sync: Updating visual elements (controller models, beams)
Post-Sync: Handling input (button presses, menu logic)

XR Coordinate Systems in Omniverse Spatial: Real World, USD Stage, and Virtual World#

XR involves multiple coordinate systems that must be understood for correct positioning.

Coordinate System Types#

1. Real World Coordinates#

Physical space where user stands/moves.

Properties:

Origin: Physical floor or play area center
Units: Meters
Axes: Follows XR runtime convention

Access:

coord_system = xr_core.get_coordinate_system()
meters_per_unit = coord_system.meters_per_unit
up_axis = coord_system.up_axis  # 'y' or 'z'

2. USD Stage Coordinates#

Virtual scene space.

Properties:

Origin: Scene-defined
Units: Configured per stage (cm, m, etc.)
Axes: Y-up or Z-up

Access:

stage_coord_system = xr_core.get_stage_coordinate_system()
stage_meters_per_unit = stage_coord_system.meters_per_unit

3. Virtual World Coordinates#

Real-world coordinates transformed into USD stage.

Transformation Chain:

Physical Device → Real World Coords → Anchor Transform → USD Stage Coords

Anchoring#

The anchor connects real-world and virtual coordinates.

Anchor Modes:

# Anchor to scene origin (default)
settings.set("/persistent/xr/profile/vr/anchorMode", "scene origin")

# Anchor to active camera
settings.set("/persistent/xr/profile/vr/anchorMode", "active camera")

# Anchor to custom USD prim
settings.set("/persistent/xr/profile/vr/anchorMode", "custom anchor")
settings.set("/xr/profile/vr/stage/customAnchor", "/World/MyAnchor")

Programmatic Anchoring:

# Set anchor to specific prim
xr_core.schedule_set_stage_anchor("/World/AnchorPrim")

# Get current anchor
anchor_path = xr_core.get_stage_anchor_prim_path()

# Detach anchor (return to default)
xr_core.detach_stage_anchor()

Transform Utilities#

Convert between coordinate systems:

# Get device pose in different systems
device = xr_core.get_input_device("/user/hand/left")

# Raw pose (device coordinates)
raw_pose = device.get_raw_pose()

# Real-world pose (meters, filtered)
world_pose = device.get_pose()

# Virtual world pose (USD stage coordinates)
virtual_pose = device.get_virtual_world_pose()

# Extract position from pose
position = virtual_pose.ExtractTranslation()  # Gf.Vec3d in stage units

USD Integration (XRUsdLayer)#

XRCore uses a specialized USD layer system to manage XR UI elements.

Why XRUsdLayer?#

Challenges:

USD composition is slow for real-time updates
Controllers must update every frame with minimal latency
Need to bypass USD’s layer stack for performance

Solution:

XRUsdLayer uses USD-RT (Real-Time) for fast updates
Manages objects like controllers, beams, teleport arcs
Handles coordinate system conversions automatically

Creating USD Layers#

# Create XR UI layer on session layer
xr_usd_layer = xr_core.create_xr_usd_layer(
    usd_path="/_xr/gui",  # Path in USD stage
    meters_per_unit=0.01,  # 1cm units
    up_axis='y'  # Y-up
)

# Access from components
from omni.kit.xr.core import XRUsdLayerManager

manager = XRUsdLayerManager.get_singleton()
controllers_layer = manager.get_usd_layer("controllers")

Managed Objects#

XRUsdLayer manages special objects efficiently:

Assets: Controller models, custom objects
Beams: Ray visualization for selection
Teleport Arcs: Curved arcs for locomotion
Links: Parent objects to devices
Transforms: Positioned objects in 3D

Example:

# Ensure device prim path exists
device_path = xr_usd_layer.ensure_device_prim_path("/user/hand/left")

# Add controller model
xr_usd_layer.add_asset(
    path=f"{device_path}/controller",
    group="controllers",
    file_path="path/to/controller.usd",
    visible=True
)

# Add selection beam
xr_usd_layer.add_beam(
    path=f"{device_path}/beam",
    group="tools",
    material_reference="/World/Materials/BeamMat",
    max_length=1000.0,  # 10m in cm
    tube_radius=0.5  # 0.5cm
)

XR Settings#

Settings control all aspects of XR configuration.

Settings Hierarchy#

/persistent/
└── xr/
    └── profile/
        ├── vr/            # VR profile saved settings
        ├── ar/            # AR profile saved settings
        └── tabletar/      # TabletAR profile saved settings

/xr/
├── defaults/              # System defaults (set in .kit or extension.toml)
└── profile/
    ├── vr/                # VR profile transient settings
    ├── ar/                # AR profile transient settings
    └── tabletar/          # TabletAR profile transient settings

The XR system reads from /persistent/xr/profile/<profile>/<setting> first (the user’s saved preference). If no value is present — such as after --reset-user clears user.config.json — it falls back to /defaults/xr/profile/<profile>/<setting>.

In .kit files, always use defaults. to set XR profile preferences, not persistent.:

[settings]
# Correct: survives --reset-user
defaults.xr.profile.ar.anchorMode = "scene origin"

# Avoid: cleared by --reset-user, overwritten by user.config.json on startup
# persistent.xr.profile.ar.anchorMode = "scene origin"

Accessing Settings#

from omni.kit.xr.core import XRSettings

xr_settings = XRSettings.get_singleton()

# Get setting
quality = xr_settings.get_setting("/persistent/xr/profile/vr/renderQuality")

# Set setting
xr_settings.set_setting("/persistent/xr/profile/vr/tooltips/visible", True)

# Use profile helper paths
profile = xr_core.get_profile("vr")
persistent_path = profile.get_persistent_path()  # "/persistent/xr/profile/vr"

Performance Considerations#

Foveated Rendering#

Renders high quality only where user is looking:

# Enable warped foveation
settings.set("/persistent/xr/profile/vr/foveation/mode", "warped")

# Configure foveation
settings.set("/persistent/xr/profile/vr/foveation/warped/resolutionMultiplier", 0.5)
settings.set("/persistent/xr/profile/vr/foveation/warped/insetSize", 0.3)

Quality Presets#

# Performance mode
settings.set("/persistent/xr/profile/vr/renderQuality", "performance")

# Balanced mode
settings.set("/persistent/xr/profile/vr/renderQuality", "balanced")

# Quality mode
settings.set("/persistent/xr/profile/vr/renderQuality", "quality")

Resolution Scaling#

# Reduce render resolution (0.1 to 2.0)
settings.set("/persistent/xr/profile/vr/render/resolutionMultiplier", 0.8)

Next Steps#

Input & Interaction – Work with controllers and input
Event System – Handle XR events
Scene Integration – Deep dive into XRUsdLayer
Settings Reference – Complete settings documentation

Key Takeaways

XRCore singleton is the main entry point
Profiles configure different XR experiences (VR/AR/Tablet)
Frame scheduling uses 2-frame latency for performance
Multiple coordinate systems must be understood
XRUsdLayer provides fast USD integration
Settings control all configuration aspects

Omniverse Spatial Core Concepts: XRCore, Profiles, Coordinate Systems, and Frame Scheduling#

XR System Architecture#

XRCore Singleton#

Accessing XRCore#

Key Responsibilities#

Profile System#

Available Profiles#

Profile Lifecycle#

Checking Profile Status#

System Plugins#

OpenXR Plugin#

SimulatedXR Plugin#

Frame Scheduling#

The Two-Frame Pipeline#

Frame Lifecycle#

Scheduling Events#

XR Coordinate Systems in Omniverse Spatial: Real World, USD Stage, and Virtual World#

Coordinate System Types#

1. Real World Coordinates#

2. USD Stage Coordinates#

3. Virtual World Coordinates#

Anchoring#

Transform Utilities#

USD Integration (XRUsdLayer)#

Why XRUsdLayer?#

Creating USD Layers#

Managed Objects#

XR Settings#

Settings Hierarchy#

Accessing Settings#

Subscribe to Changes#

Performance Considerations#

Foveated Rendering#

Quality Presets#

Resolution Scaling#

Next Steps#