Extension: omni.kit.ui_test-1.4.2

Documentation Generated: Oct 09, 2025

Overview

The omni.kit.ui_test extension provides automated UI testing capabilities for Omniverse Kit applications. This extension enables developers to programmatically interact with UI elements through mouse and keyboard simulation, widget discovery, and menu navigation to create comprehensive automated test suites.

Concepts

Widget Discovery and References

The extension uses omni.ui_query under the hood to locate UI elements through path-based queries. Found widgets are wrapped in WidgetRef objects that provide both reference information and testing methods. This approach allows tests to interact with UI elements without directly manipulating the underlying widget instances.

Human-like Interaction

All interaction functions include human_delay_speed parameters to simulate realistic user behavior. The extension provides human_delay specifically for adding natural pauses between actions, making automated tests appear more like actual user interactions rather than instantaneous programmatic operations.

Asynchronous Operations

All testing operations are asynchronous, allowing proper integration with Kit’s frame-based update system. The wait_n_updates function provides frame-accurate timing control for coordinating test actions with application state changes.

Key Components

Widget Discovery Functions

The find, find_all, and find_first functions locate UI elements using query paths. These functions return WidgetRef objects that encapsulate both the widget reference and testing capabilities. The find_with_retry function adds robustness by attempting widget discovery multiple times across application frames.

Input Simulation

Mouse simulation includes emulate_mouse_click, emulate_mouse_move, emulate_mouse_drag_and_drop, and emulate_mouse_scroll for comprehensive cursor interaction. Keyboard simulation provides emulate_keyboard_press, emulate_char_press, and emulate_key_combo for text input and key combination testing.

WidgetRef Class

The WidgetRef class serves as the primary interface for widget testing operations. It provides properties like position, size, center, and model, along with methods like click(), input(), drag_and_drop(), and widget-specific discovery functions. This class bridges the gap between widget identification and test actions.

KeyDownScope Context Manager

KeyDownScope provides an async context manager for holding keys in the pressed state throughout a code block. This is particularly useful for modifier key combinations or testing scenarios that require sustained key presses during other operations.

Menu Navigation

Menu testing capabilities include menu_click for navigating hierarchical menus using path strings, get_context_menu for retrieving context menu structures, and select_context_menu for interacting with right-click menus. The MenuRef class provides specialized menu widget handling.

Usage Examples

Basic Widget Interaction

import omni.kit.ui_test as ui_test

# Find a widget and perform actions
stage_window = ui_test.find("Stage//Frame/**/ScrollingFrame/TreeView[*].visible==True")
await stage_window.click()

# Input text into a field
text_field = ui_test.find("**/StringField[*]")
await text_field.input("Hello World")

Menu Navigation

# Navigate through menu hierarchy
await ui_test.menu_click("File/New")

# Select context menu items
await ui_test.select_context_menu("Copy/All")