Overview#

The omni.services.livestream.webrtc extension provides a web interface for accessing WebRTC streams from Omniverse Kit applications. It serves a simple streaming client that users can access directly through their web browser.

Purpose#

This extension eliminates the complexity of WebRTC streaming setup by providing:

  1. Web Streaming Interface: Simple web client interface with WebRTC connectivity

  2. Auto-Configuration: Automatically detects and configures available streams

  3. Zero-Setup Access: Users simply navigate to a URL to start streaming

Architecture#

The extension consists of these key components:

  • FastAPI Web Server: Serves the streaming interface and provides REST APIs

  • Dynamic Stream Discovery: Automatically detects primary, spectator, and AOV streams

  • Web Streaming Client: Feature-rich HTML/CSS/JavaScript interface using the @nvidia/omniverse-webrtc-streaming-library

  • Configuration API: RESTful endpoint for retrieving stream configurations

Dependencies: This extension requires omni.kit.livestream.app and/or omni.kit.livestream.aov to be enabled for stream configuration discovery.

Key Features#

🎯 Streaming Capabilities#

  • Real WebRTC Connections: Full peer-to-peer streaming using industry-standard WebRTC

  • Multiple Stream Support: Automatic detection of primary, spectator, and AOV streams

  • Dynamic Resolution Control: Real-time resolution changes during active streams

  • Pause/Resume Functionality: Stream control without disconnecting

🔧 Configuration & Discovery#

  • Auto-Detection: Automatically discovers server IP and available stream ports

  • Smart Stream Selection: Dropdown interface for choosing between multiple streams

  • Settings Integration: Reads stream configurations directly from Kit settings

  • Dynamic Resize Support: Honors per-stream allowDynamicResize settings

🎨 User Interface#

  • Modern Design: Clean, responsive interface with professional styling

  • Mobile-Friendly: Works on desktop, tablet, and mobile devices

  • Real-Time Status: Live connection status and error reporting

  • Intuitive Controls: Simple connect/disconnect/pause/resume workflow

Usage#

Quick Start#

  1. Enable the extension in your Kit application:

    [dependencies]
"omni.services.livestream.webrtc" = {}

  2. Configure streaming (example):

    [settings]
exts."omni.kit.livestream.app".primaryStream.streamType = "webrtc"
exts."omni.kit.livestream.app".primaryStream.signalPort = 49100
exts."omni.services.transport.server.http".port = 8011

  3. Access the interface at: http://<server-ip>:8011

Accessing Streams#

Users can access the streaming interface by navigating to:

  • Local Access: http://localhost:8011

  • Remote Access: http://<server-ip>:8011 ( must be directly accessible over the network from the client machine)

The interface will automatically:

  • Detect the server IP address

  • Discover all configured streams

  • Present them in an easy-to-use dropdown menu

Stream Types Supported#

The extension automatically detects and presents:

  • Primary App Streams: Main application rendering output

  • Spectator App Streams: Secondary application views (indexed)

  • AOV Streams: Arbitrary Output Variables for specific rendering layers

Configuration#

HTTP Server Settings#

[settings]
# Web interface port
exts."omni.services.transport.server.http".port = 8011

Stream Configuration Examples#

[settings]
# Primary stream
exts."omni.kit.livestream.app".primaryStream.streamType = "webrtc"
exts."omni.kit.livestream.app".primaryStream.signalPort = 49100
exts."omni.kit.livestream.app".primaryStream.allowDynamicResize = true

# Spectator streams
exts."omni.kit.livestream.app".spectatorStream.0.streamType = "webrtc"
exts."omni.kit.livestream.app".spectatorStream.0.signalPort = 49200

# AOV streams (automatically detected)
exts."omni.kit.livestream.aov"."{aov}".spectatorStream.0.streamType = "webrtc"

REST API#

The extension provides the following REST endpoints:

Stream Configuration#

GET /api/stream-config

Returns JSON with all detected stream configurations:

{
  "streams": [
    {
      "name": "Primary App Stream",
      "type": "primary",
      "port": 49100,
      "streamType": "webrtc",
      "allowDynamicResize": true,
      "description": "Main application stream (webrtc)"
    }
  ]
}

Web Interface#

GET /
GET /index.html

Serves the main streaming interface.

Static Assets#

GET /static/*

Serves web assets including the Omniverse WebRTC streaming library.

Integration#

This extension works seamlessly with:

  • omni.kit.livestream.webrtc: Provides the actual WebRTC streaming functionality

  • omni.kit.livestream.app: Configures application streaming settings

  • omni.kit.livestream.aov: Enables Arbitrary Output Variable streaming

  • omni.services.core: FastAPI service framework

  • omni.services.transport.server.http: HTTP transport layer

Development#

Web Interface Technology Stack#

  • HTML5: Semantic structure and video/audio elements

  • CSS3: Modern styling with responsive design and light theme

  • Vanilla JavaScript: Real-time WebRTC connectivity without framework dependencies

  • @nvidia/omniverse-webrtc-streaming-library: Omniverse WebRTC streaming library

Key Components#

  • AppStreamer: Core streaming class from the web streaming library

  • Stream Discovery: Dynamic detection of Kit stream configurations

  • Connection Management: Handles connect/disconnect/pause/resume lifecycle

  • Resolution Control: Real-time resolution changes with stream capability awareness

Extending the Interface#

The web interface can be customized by modifying:

  • web/index.html: Main interface structure and functionality

  • Stream discovery logic in extension.py

  • REST API endpoints for additional functionality

HTTPS Configuration#

For production deployments and enhanced security, you can configure the service to operate over HTTPS instead of HTTP.

Please see https://docs.omniverse.nvidia.com/services/latest/transports/transport_http.html

Troubleshooting#

Common Issues#

No streams detected:

  • Ensure omni.kit.livestream.app or omni.kit.livestream.aov is enabled

  • Verify stream configuration settings for primary and spectator streams

  • Check that signaling ports are configured for each enabled stream

Connection failures:

  • Confirm WebRTC server is running in Kit

  • Check network connectivity between machines

  • Verify firewall settings for signaling ports

  • Check browser console for specific error messages

  • Ensure server IP is accessible from client machine

  • Verify no crypto-related errors in browser console