Viewport API

The ViewportAPI object is used to access and control aspects of the render, who will then push images into the backing texture. It is accessed via the ViewportWidget.viewport_api property which returns a Python weakref.proxy object. The use of a weakref is done in order to make sure that a Renderer and Texture aren’t kept alive because client code is keeping a reference to one of these objects. That is: the lifetime of a Viewport is controlled by the creator of the Viewport and cannot be extended beyond the usage they expect.

With our simple omni.kit.viewport.stage_preview example, we’ll now run through some cases of directly controlling the render, similar to what the Content context-menu does.

from omni.kit.viewport.stage_preview.window import StagePreviewWindow
# Viewport widget can be setup to fill the frame with 'fill_frame', or to a constant resolution (x, y).
resolution = 'fill_frame' or (640, 480) # Will use 'fill_frame'

# Give the Window a unique name, the window-size, and forward the resolution to the Viewport itself
preview_window = StagePreviewWindow('My Stage Preview', usd_context_name='MyStagePreviewWindowContext',
                                    window_width=640, window_height=480, resolution=resolution)

# Open a USD file in the UsdContext that the preview_window is using
preview_window.open_stage('http://omniverse-content-production.s3-us-west-2.amazonaws.com/Samples/Astronaut/Astronaut.usd')

# Save an easy-to-use reference to the ViewportAPI object
viewport_api = preview_window.preview_viewport.viewport_api

Camera

The camera being used to render the image is controlled with the camera_path property. It will return and is settable with a an Sdf.Path. The property will also accept a string, but it is slightly more efficient to set with an Sdf.Path if you have one available.

print(viewport_api.camera_path)
# => '/Root/LongView'

# Change the camera to view through the implicit Top camera
viewport_api.camera_path = '/OmniverseKit_Front'

# Change the camera to view through the implicit Perspective camera
viewport_api.camera_path = '/OmniverseKit_Persp'

Filling the canvas

Control whether the texture-resolution should be adjusted based on the parent objects ui-size with the fill_frame property. It will return and is settable with a bool.

print('viewport_api.fill_frame)
# => True

# Disable the automatic resolution
viewport_api.fill_frame = False

# And give it an explicit resolution
viewport_api.resolution = (640, 480)

Resolution

Resolution of the underlying ViewportTexture can be queried and set via the resolution property. It will return and is settable with a tuple representing the (x, y) resolution of the rendered image.

print(viewport_api.resolution)
# => (640, 480)

# Change the render output resolution to a 512 x 512 texture 
viewport_api.resolution = (512, 512)

SceneView

For simplicity, the ViewportAPI can push both view and projection matrices into an omni.ui.scene.SceneView.model. The use of these methods are not required, rather provided a convenient way to auto-manage their relationship (particularly for stage independent camera, resolution, and widget size changes), while also providing room for future expansion where a SceneView may be locked to last delivered rendered frame versus the current implementation that pushes changes into the Renderer and the SceneView simultaneously.

add_scene_view

Add an omni.ui.scene.SceneView to the list of models that the Viewport will notify with changes to view and projection matrices. The Viewport will only retain a weak-reference to the provided SceneView to avoid extending the lifetime of the SceneView itself. A remove_scene_view is available if the need to dynamically change the list of models to be notified is necessary.

scene_view = omni.ui.scene.SceneView()
# Pass the real object, as a weak-reference will be retained
viewport_api.add_scene_view(scene_view)

remove_scene_view

Remove the omni.ui.scene.SceneView from the list of models that the Viewport will notify from changes to view and projection matrices. Because the Viewport only retains a weak-reference to the SceneView, calling this explicitly isn’t mandatory, but can be used to dynamically change the list of SceneView objects whose model should be updated.

viewport_api.remove_scene_view(scene_view)

Additional Accessors

The ViewportAPI provides a few additional read-only accessors for convenience:

usd_context_name

viewport_api.usd_context_name => str
# Returns the name of the UsdContext the Viewport is bound to.

usd_context

viewport_api.usd_context => omni.usd.UsdContext
# Returns the actual UsdContext the Viewport is bound to, essentially `omni.usd.get_context(self.usd_context_name)`.

stage

viewport_api.stage => Usd.Stage
# Returns the `Usd.Stage` the ViewportAPI is bound to, essentially: `omni.usd.get_context(self.usd_context_name).get_stage()`.

projection

viewport_api.projection=> Gf.Matrix4d
# Returns the Gf.Matrix4d of the projection.  This may differ than the projection of the `Usd.Camera` based on how the Viewport is placed in the parent widget.

transform

viewport_api.transform => Gf.Matrix4d
# Return the Gf.Matrix4d of the transform for the `Usd.Camera` in world-space.

view

viewport_api.view => Gf.Matrix4d
# Return the Gf.Matrix4d of the inverse-transform for the `Usd.Camera` in world-space.

time

viewport_api.time => Usd.TimeCode
#  Return the Usd.TimeCode that the Viewport is using.

Space Mapping

The ViewportAPI natively uses NDC coordinates to interop with the payloads from omni.ui.scene gestures. To understand where these coordinates are in 3D-space for the ViewportTexture, two properties will can be used to get a Gf.Matrix4d to convert to and from NDC-space to World-space.

origin = (0, 0, 0)
origin_screen = viewport_api.world_to_ndc.Transform(origin)
print('Origin is at {origin_screen}, in screen-space')

origin = viewport_api.ndc_to_world.Transform(origin_screen)
print('Converted back to world-space, origin is {origin}')

world_to_ndc

viewport_api.world_to_ndc => Gf.Matrix4d
# Returns a Gf.Matrix4d to move from World/Scene space into NDC space 

ndc_to_world

viewport_api.ndc_to_world => Gf.Matrix4d
# Returns a Gf.Matrix4d to move from NDC space into World/Scene space 

World queries

While it is useful being able to move through these spaces is useful, for the most part what we’ll be more interested in using this to inspect the scene that is being rendered. What that means is that we’ll need to move from an NDC coordinate into pixel-space of our rendered image.

A quick breakdown of the three spaces in use:

The native NDC space that works with omni.ui.scene gestures can be thought of as a cartesian space centered on the rendered image.

We can easily move to a 0.0 - 1.0 uv space on the rendered image.

And finally expand that 0.0 - 1.0 uv space into pixel space.

Now that we know how to map between the ui and texture space, we can use it to convert between screen and 3D-space. More importantly, we can use it to actually query the backing ViewportTexture about objects and locations in the scene.

request_query

viewport_api.request_query(pixel_coordinate: (x, y), query_completed: callable) => None
# Query a pixel co-ordinate with a callback to be executed when the query completes


# Get the mouse position from within an omni.ui.scene gesture
mouse_ndc = self.sender.gesture_payload.mouse

# Convert the ndc-mouse to pixel-space in our ViewportTexture
mouse, in_viewport = viewport_api.map_ndc_to_texture_pixel(mouse_ndc)

# We know immediately if the mapping is outside the ViewportTexture
if not in_viewport:
    return

# Inside the ViewportTexture, create a callback and query the 3D world
def query_completed(path, pos, *args):
    if not path:
        print('No object')
    else:
        print(f"Object '{path}' hit at {pos}")

viewport_api.request_query(mouse, query_completed)

Asynchronous waiting

Some properties and API calls of the Viewport won’t take effect immediately after control-flow has returned to the caller. For example, setting the camera-path or resolution won’t immediately deliver frames from that Camera at the new resolution. There are two methods available to asynchrnously wait for the changes to make their way through the pipeline.

wait_for_render_settings_change

Asynchronously wait until a render-settings change has been ansorbed by the backend. This can be useful after changing the camera, resolution, or render-product.

my_custom_product = '/Render/CutomRenderProduct'
viewport_api.render_product_path = my_custom_product
settings_changed = await viewport_api.wait_for_render_settings_change()
print('settings_changed', settings_changed)

wait_for_rendered_frames

Asynchronously wait until a number of frames have been delivered. Default behavior is to wait for deivery of 1 frame, but the function accepts an optional argument for an additional number of frames to wait on.

await viewport_api.wait_for_rendered_frames(10)
print('Waited for 10 frames')

Capturing Viewport frames

To access the state of the Viewport’s next frame and any AOVs it may be rendering, you can schedule a capture with the schedule_capture method. There are a variety of default Capturing delegates in the omni.kit.widget.viewport.capture that can handle writing to disk or accesing the AOV’s buffer.

Because capture is asynchronous, it will likely not have occured when control-flow has returned to the caller of schedule_capture. To wait until it has completed, the returned object has a wait_for_result method that can be used.

schedule_capture

viewport_api.schedule_capture(capture_delegate: Capture): -> Capture

from omni.kit.widget.viewport.capture import FileCapture

capture = viewport_api.schedule_capture(FileCapture(image_path))
captured_aovs = await capture.wait_for_result()
if captured_aovs:
  print(f'AOV "{captured_aovs[0]}" was written to "{image_path}"')
else:
  print(f'No image was written to "{image_path}"')