Extension: omni.kit.ui_test-1.5.3

Documentation Generated: Aug 27, 2026

Overview

The omni.kit.ui_test extension provides automated UI testing capabilities for Omniverse Kit applications. This extension enables developers to programmatically interact with UI elements through mouse and keyboard simulation, widget discovery, and menu navigation to create comprehensive automated test suites.

Concepts

Widget Discovery and References

The extension uses omni.ui_query under the hood to locate UI elements through path-based queries. Found widgets are wrapped in WidgetRef objects that provide both reference information and testing methods. This approach allows tests to interact with UI elements without directly manipulating the underlying widget instances.

Stale Widget Detection

A WidgetRef becomes stale when its underlying widget is no longer in the UI tree — the containing window was destroyed, the widget was removed from its parent, or the panel was rebuilt. Accessing the widget, model, position, size, or center properties of a stale ref logs a big multi-line warning (it does not raise) and still returns the now-detached widget. Use the is_stale property to probe non-destructively. When a test does an action that may rebuild the UI (a click, a model mutation, a drag-drop, an undo), re-query the widget afterwards rather than reusing the old WidgetRef.

Human-like Interaction

All interaction functions include human_delay_speed parameters to simulate realistic user behavior. The extension provides human_delay specifically for adding natural pauses between actions, making automated tests appear more like actual user interactions rather than instantaneous programmatic operations.

Asynchronous Operations

All testing operations are asynchronous, allowing proper integration with Kit’s frame-based update system. The wait_n_updates function provides frame-accurate timing control for coordinating test actions with application state changes.

Key Components

Widget Discovery Functions

The find, find_all, and find_first functions locate UI elements using query paths. These functions return WidgetRef objects that encapsulate both the widget reference and testing capabilities. The find_with_retry function adds robustness by attempting widget discovery multiple times across application frames.

Input Simulation

Mouse simulation includes emulate_mouse_click, emulate_mouse_move, emulate_mouse_drag_and_drop, and emulate_mouse_scroll for comprehensive cursor interaction. Keyboard simulation provides emulate_keyboard_press, emulate_char_press, and emulate_key_combo for text input and key combination testing.

WidgetRef Class

The WidgetRef class serves as the primary interface for widget testing operations. It provides properties like position, size, center, and model, along with methods like click(), input(), drag_and_drop(), and widget-specific discovery functions. This class bridges the gap between widget identification and test actions.

KeyDownScope Context Manager

KeyDownScope provides an async context manager for holding keys in the pressed state throughout a code block. This is particularly useful for modifier key combinations or testing scenarios that require sustained key presses during other operations.

Menu Navigation

Menu testing capabilities include menu_click for navigating hierarchical menus using path strings, get_context_menu for retrieving context menu structures, and select_context_menu for interacting with right-click menus. The MenuRef class provides specialized menu widget handling.

Usage Examples

Basic Widget Interaction

import omni.kit.ui_test as ui_test

# Find a widget and perform actions
stage_window = ui_test.find("Stage//Frame/**/ScrollingFrame/TreeView[*].visible==True")
await stage_window.click()

# Input text into a field
text_field = ui_test.find("**/StringField[*]")
await text_field.input("Hello World")

Menu Navigation

# Navigate through menu hierarchy
await ui_test.menu_click("File/New")

# Select context menu items
await ui_test.select_context_menu("Copy/All")

Handling Stale Widgets

When an action causes the UI to rebuild, re-query the widget instead of reusing the previous WidgetRef:

# Wrong — the click may rebuild the panel, leaving `btn` detached.
btn = ui_test.find("MyWindow//Frame/**/Button[*].text=='Apply'")
await btn.click()
value = btn.widget.model.get_value_as_bool()  # logs a big multi-line stale-widget warning

# Right — capture state you need before the action, or re-find after.
btn = ui_test.find("MyWindow//Frame/**/Button[*].text=='Apply'")
model = btn.widget.model  # captured while widget is fresh
await btn.click()
value = model.get_value_as_bool()  # model survives the rebuild

# Or re-find after the action:
await ui_test.find("MyWindow//Frame/**/Button[*].text=='Apply'").click()
btn = ui_test.find("MyWindow//Frame/**/Button[*].text=='Apply'")
value = btn.widget.model.get_value_as_bool()

Use is_stale to probe without raising:

ref = ui_test.find(some_path)
# ... actions that may rebuild UI ...
if ref.is_stale:
    ref = ui_test.find(some_path)