Is this page helpful?

OV Web RTC Library v6.8.2#

This package provides functionality for managing Omniverse Kit applications streaming from GFN, on a local machine, or from a container instance with a provided URL.

Note

Versioned documentation: The complete API documentation is bundled with the published package. To read the docs that match the version you have installed, open node_modules/@nvidia/ov-web-rtc/docs/index.html in a browser after installing.

Usage#

The AppStreamer class is the main entry point into this library. You can use it to connect to, pause, unpause, and terminate a streaming Omniverse Kit application session. You can also use it to send and receive custom messages to and from the streaming Kit application. The Kit application can stream from GFN, NVCF, OVC, or an ad hoc development environment.

Scoping NPM#

The correct scope must be defined in your .npmrc file before installing.

@nvidia:registry=https://edge.urm.nvidia.com/artifactory/api/npm/omniverse-client-npm/

Installation#

npm install @nvidia/ov-web-rtc

Importing#

import { AppStreamer } from '@nvidia/ov-web-rtc'

GFN#

If connecting to an application running on GFN, you must import GFN separately. The recommended method to import GFN into a project is by sourcing the CDN in the project index.html file, as shown below:

<!DOCTYPE html>
<html lang="en">
    <body>
        <script src="https://sdk.nvidia.com/gfn/client-sdk/1.x/gfn-client-sdk.js"></script>
    </body>
</html>

Creating an AppStreamer#

Construct an AppStreamer instance and drive the session through its lifecycle methods — connect, start, stop, terminate — plus sendMessage for custom messages:

const stream = new AppStreamer();

Each instance owns its own underlying stream. The examples below use this stream instance.

The static form (AppStreamer.connect(...), AppStreamer.terminate(), and so on) is deprecated but still works for backward compatibility: it forwards to a single shared instance — race-prone when more than one stream is active — and each call logs a one-time deprecation warning. It will be removed in a future major release. Construct an instance with new AppStreamer() instead.

Examples#

Connecting#

The following code examples illustrate how to manage streaming connections directly to a container, as well as infrastructure environments such as GFN and NVCF.

Set up the Configuration#

Direct Configuration#

Configuration for a Kit application running in a directly accessible container, either local or remote:

// This example server is localhost, but can be a valid remote IP as well.

const streamParams: DirectConfig = {
    streamSource : StreamType.DIRECT,
    logLevel     : LogLevel.WARN,
    streamConfig : {
        server        : '<some ip address>',
        width         : 1920,
        height        : 1028,
        fps           : 60,
        onStart       : (message: StreamEvent) => {console.info('Stream started')},
        onStop        : (message: StreamEvent) => {console.info('Stream stopped')},
        onStreamStats : (message: StreamEvent) => {console.info('Stream stats')}
    }
};

Configure NVCF#

Configuration for a Kit application running on NVCF:

// This example server is localhost, but can be a valid remote IP as well.

const streamParams: NVCFConfig = {
    streamSource : StreamType.NVCF,
    logLevel     : LogLevel.WARN,
    streamConfig : {
        signalingPort   : <some port number>
        signalingServer : '<some URL>',
        signalingPath   : '<some path>'
        signalingQuery  : ''<some query>'
        width           : 1920,
        height          : 1028,
        fps             : 60,
        onStart         : (message: StreamEvent) => {console.info('Stream started')},
        onStop          : (message: StreamEvent) => {console.info('Stream stopped')},
        onStreamStats   : (message: StatsEvent) => {console.info('Stream stats')}
    }
};

Configure GFN#

Configuration for a Kit app running on GFN:

const streamParams: GFNConfig = {
    streamSource : StreamType.GFN,
    logLevel     : LogLevel.WARN,
    streamConfig : {
        // GFN will be resolved by the script source discussed above.
        GFN             : GFN,
        catalogClientId : <your value>,
        clientId        : <your value>,
        cmsId           : <your value>,
        onStart         : (message: StreamEvent) => {console.info('Stream started')},
        onStop          : (message: StreamEvent) => {console.info('Stream stopped')},
        onStreamStats   : (message: StreamEvent) => {console.info('Stream stats')}
    }
};

Connect & Disconnect#

Connect using Config params#

The library supports connections to Kit applications streaming from multiple infrastructure options. The options are of type StreamType. They allow you to change which infrastructure to connect to with only minor edits to the configuration. Below are examples of configurations for each streaming type.

Using the streamParams config:

stream.connect(streamParams)
    .then((result: StreamEvent) => {
       // The connection request was successful. The onStart
       // callback will fire when the stream is ready.
       console.info(result);
    })
    .catch((error: StreamEvent) => {
       // The connection request has failed.
       console.error(error);
    });

Disconnect#

Call terminate() on the same instance you connected with:

stream.terminate()
    .then((result: StreamEvent) => {
          // Request has been made successfully. The onStop
          // callback will fire when termination is complete.
          console.info(result));
    })
    .catch((error: StreamEvent) => {
          // The terminate request has failed.
          console.error(error));
    });

Callbacks#

All of the callbacks are optional, but there are a few that are very helpful.

onStreamStatusChange()#

Use onStreamStatusChange for lifecycle updates. The callback receives the new StreamStatus; use stream.streamStatus when you only need the current value. onUpdate reports operation events and does not cover every status transition.

onStreamStatusChange: (status: StreamStatus) => {
    console.log('Stream status:', StreamStatus[status]);
}

During connection retries, the reported status remains STARTING until the stream connects or all retry attempts fail.

onStart()#

Define your onStart callback in streamParams to determine the success of the connection request.

onStart(message: StreamEvent) : void {
    if ( message.status === EventStatus.SUCCESS ) {
        // The stream is connected and ready.
        console.info('onStart:', message);
    }
    else if ( message.status === EventStatus.WARNING ) {
        // There may be an issue with the stream connection.
        console.warn('onStart:', message);
    }
    else if ( message.status === EventStatus.ERROR ) {
        // The connection has failed.
        console.info('onStart:', message);
    }
}

onStop()#

Define your onStop callback message to determine unexpected and successful stream disconnections.

onStop(message: StreamEvent) : void {
    if ( message.action === EventAction.TERMINATE &&
          message.status === EventStatus.ERROR ) {
          // The connected stream has been disconnected unexpectedly.
          console.error('onStop:', message);
    }
    else if ( message.action === EventAction.TERMINATE &&
                message.status === EventStatus.SUCCESS ) {
          // A request to disconnect the stream was successful -
          // the stream has been disconnected.
          console.info('onStop:', message);
    }
}

onStreamStats()#

The onStreamStats callback, if passed, will be called at regular intervals to provide information and performance metrics about the connected stream.

onStreamStats(message: StatsEvent) : void {
    console.info('Stream stats:', message.stats);
}

Cursor Support#

When connected to a Kit app that uses cursor changes to communicate tool state (for example, a crosshair for a selection tool or an I-beam over text inputs), the SDK automatically reflects the Kit cursor on the stream viewport element — no configuration needed.

Supported Cursors#

The SDK applies the cursors below to the <video> element automatically; no onCursor callback is required. They are all members of the exported StreamCursorType enum, which is named after the web (CSS) cursor each one is shown as. Kit sends each cursor by its own name, listed in the first column, and that name is passed to onCursor as kitCursorName for reference. The first six are Kit’s standard shapes. Grab_open, Grab_close and Orbit are custom cursors that Kit registers by name through omni.kit.window.cursor; the SDK handles them itself so they are in the same list.

Kit name	`StreamCursorType`	Registered by	Applied as
`Arrow`	`DEFAULT`	Kit (standard shape)	`default`
`IBeam`	`TEXT`	Kit (standard shape)	`text`
`Hand`	`POINTER`	Kit (standard shape)	`pointer`
`Crosshair`	`CROSSHAIR`	Kit (standard shape)	`crosshair`
`HorizontalResize`	`EW_RESIZE`	Kit (standard shape)	`ew-resize`
`VerticalResize`	`NS_RESIZE`	Kit (standard shape)	`ns-resize`
`Grab_open`	`GRAB`	`omni.kit.window.cursor` (default-registered)	native CSS `grab`
`Grab_close`	`GRABBING`	`omni.kit.window.cursor` (default-registered)	native CSS `grabbing`
`Orbit`	`ORBIT`	`omni.kit.viewport.navigation.camera_manipulator`	bundled image, with a `move` fallback

Each value of StreamCursorType is the CSS cursor value the SDK sets, except ORBIT ('orbit'), which has no CSS equivalent. Grab_open and Grab_close use the browser’s native grab and grabbing cursors, so they look like the standard cursors. Their artwork varies by platform, and iPadOS does not draw these keywords as hands. If you target iPadOS, handle them in onCursor. Orbit is a bundled image: the SDK applies it as image-set(url(...) 2x), so the browser draws the 41 px PNG at about 20 CSS px using the original pixels, which stays sharp on high-density displays. The hotspot is the image center (10 10), and move is the fallback.

If wired, onCursor fires for every cursor in the table with { streamCursor, kitCursorName }, after the SDK has applied the cursor. Override by setting video.style.cursor yourself, or read video.style.cursor to see what was applied.

Named cursors depend on Kit announcing them. The SDK can only apply Grab_open, Grab_close and Orbit when Kit sends their names. If Kit’s windowing plugin fails to start (for example, no display is available), Kit does not announce these named cursors, although the six standard shapes still work. If they never appear in your stream, check the Kit log for GLFW initialization failed.

Deprecated Cursor Enums#

CursorType and StreamCursorNameType, as exported in 6.7.0, are still exported and still work, but are deprecated in favor of StreamCursorType. CursorType, StreamCursorNameType and CursorChange.cursor will be removed in v7.0. They will not gain new members, so new cursors such as GRAB, GRABBING and ORBIT are only in StreamCursorType. Code written against them keeps compiling: CursorType is the six original cursors, and the cursor in a CursorChange is still typed as it and set for those six, so change.cursor === CursorType.Text, Record<CursorType, ...> maps and CursorType.Text as a type all work as before. cursor is not set for GRAB, GRABBING or ORBIT; read streamCursor, which is set for every cursor the SDK applies. cursor is deprecated in favor of streamCursor.

Deprecated	Use instead
`CursorType.Default`	`StreamCursorType.DEFAULT`
`CursorType.Text`	`StreamCursorType.TEXT`
`CursorType.Pointer`	`StreamCursorType.POINTER`
`CursorType.Crosshair`	`StreamCursorType.CROSSHAIR`
`CursorType.HorizontalResize`	`StreamCursorType.EW_RESIZE`
`CursorType.VerticalResize`	`StreamCursorType.NS_RESIZE`
`CursorChange.cursor`	`CursorChange.streamCursor`, which is also set for `GRAB`, `GRABBING` and `ORBIT`
`StreamCursorNameType.*`	Compare `kitCursorName` with the Kit name, for example `'Hand'`, or use the `StreamCursorType` listed beside it in the table above

Migrating an onCursor handler:

// 6.7.0
onCursor: ({ cursor, kitCursorName }) => {
    if (cursor === CursorType.Text) {
        showTextCursorHint();
    }
    if (cursor === undefined) {
        // Custom cursor: set it yourself
        video.style.cursor = myCursorFor(kitCursorName);
    }
}

// > 6.7.0
onCursor: ({ streamCursor, kitCursorName }) => {
    if (streamCursor === StreamCursorType.TEXT) {
        showTextCursorHint();
    }
    if (streamCursor === undefined) {
        // Custom cursor: set it yourself
        video.style.cursor = myCursorFor(kitCursorName);
    }
}

Check any cursor === undefined test. Handlers that use it to detect a custom cursor are also true for GRAB, GRABBING and ORBIT, because cursor is only set for the six CursorType cursors, and the SDK has already applied those three. Test streamCursor === undefined instead, as in the example above.

Custom Cursors#

For any cursor name that is not in the table above — if your Kit app registers its own cursor shapes (through omni.kit.window.cursor), wire the onCursor callback in DirectConfig. It receives a CursorChange object:

Cursor kind	Example names	`streamCursor`	`kitCursorName`	What the SDK did
Supported	`Arrow`, `Hand`, `Grab_open`, `Orbit`	the `StreamCursorType` it applied	set	Applied the cursor; the callback is a notification and can override.
Other custom	`Rotate_object`, your own names	absent	set	Applied nothing; your handler must set the cursor.

The callback runs after the SDK has applied its cursor, so anything you set on video.style.cursor wins.

import { AppStreamer, StreamType, type CursorChange } from '@nvidia/ov-web-rtc';

const video = document.getElementById('remote-video') as HTMLVideoElement;

const stream = new AppStreamer();
await stream.connect({
    streamSource: StreamType.DIRECT,
    streamConfig: {
        signalingServer: '...',
        onCursor: ({ streamCursor, kitCursorName }: CursorChange) => {
            if (streamCursor === undefined) {
                // Custom cursor — SDK applied nothing, handle it here
                video.style.cursor = `url(/images/${kitCursorName}.png), auto`;
                return;
            }
            // Supported cursor — SDK already applied it; optionally override here
        },
    },
});

The SDK always applies the cursors in the supported table automatically regardless of whether onCursor is wired. Wire it only when you have custom cursors to handle.

Note

Kit version requirements: cursor support requires omni.kit.livestream.webrtc v10.7.0, omni.kit.livestream.app v10.3.0, and omni.kit.renderer.imgui v2.2.0. On older Kit builds the feature is inactive and the viewport cursor stays at its default.