Overview#

omni.kit.test is an asyncio-centric testing system designed for running and managing tests within Omniverse Kit extensions. It provides a range of asynchronous test case classes, a test suite, and utilities for filtering, reporting, and benchmarking tests. By leveraging Python’s asyncio framework, the extension enables tests to run concurrently, retry operations until success, and integrate detailed result reporting all while keeping the overall workload lightweight and efficient.

graph TD %% Inheritance relationships AsyncTestCase --> AsyncTestCaseFailOnLogError AsyncTestCase --> BenchmarkTestCase TestPopulator --> TestPopulateAll TestPopulator --> TestPopulateDisabled

Concepts#

Asynchronous Testing: Base classes like AsyncTestCase, AsyncTestSuite, and BenchmarkTestCase enable writing tests that await on non-blocking operations. This is especially useful for tests that depend on frame updates or external asynchronous events.
Test Reporting and Status: The extension defines standard status enumerations and reporting callbacks through APIs such as TestRunStatus and add_test_status_report_cb. These callbacks help integrate custom reporting components to track test progress in real time.
Dynamic Test Discovery: Via test populators (TestPopulateAll and TestPopulateDisabled) and utilities to scan for test modules, tests can be automatically discovered from enabled extensions. This ensures that both explicitly written and dynamically imported tests are properly aggregated.

Functionality#

Asynchronous Execution of Tests: Tests inherit from AsyncTestCase or its variant that fails on logged errors, ensuring that asynchronous workflows (like waiting for frame updates or retrying operations until success) integrate naturally into the test run.
Robust Test Runner: The module provides run_tests along with an asynchronous text test runner to execute tests, aggregate results, and handle retries based on configurable strategies (for instance, using strategies like no-retry or retry-on-failure).
Benchmarking Support: BenchmarkTestCase and related reporter APIs enable capturing performance-based metrics alongside typical pass/fail results, allowing comparisons through custom metrics and fingerprinting.
Test Environment Control: Through ExtTest and its associated utilities, the extension wraps test execution into separate processes with options for parallel execution, test filtering (via include, exclude, and wildcard matching settings), and collecting code coverage data.

Configuration#

Unique settings in the extension.toml file help fine-tune test execution without affecting core logic:

Settings such as runTestsAndQuit, doNotQuit, and printTestsAndQuit determine whether tests run automatically, if the application exits after tests complete, or if tests are printed to stdout.
Filters like includeTests, excludeTests, and runTestsFilter allow granular selection of tests by their unique identifiers, matching patterns typically in the format [module].[class].[method].
Parameters for extension testing such as testExtOutputPath, testExtMaxParallelProcesses, and various code coverage options provide control over logging, parallelism, and diagnostic data collection.

Relationships#

The extension depends on omni.kit.async_engine, omni.kit.loop, and omni.kit.pip_archive to support its asynchronous operations and module packaging.
It integrates with tools like TeamCity and GitLab for CI reporting, and supports additional frameworks for benchmarking and retry logic.
By automatically discovering test modules and integrating with standardized output paths, it helps maintain consistent test behaviors across all extensions within Omniverse Kit.

Overall, omni.kit.test delivers a practical solution to streamline extension testing with asynchronous capabilities, dynamic test population, and integrated reporting—making it easier for developers to ensure stability and performance across extension codebases.