Overview#

This extension wraps RTX Raycast Query, providing a simpler raycast interface into the current stage.

Class list#

  • IRaycastQuery: Interface class for raycast-query operations. The thread-safe interface supports individual asynchronous queries, polling a sequence of individual rays, and polling a sequence of ray arrays.

  • Result: Enumeration of possible raycast-query error codes.

  • RayQueryResult: Alias of rtx::raytracing::RaycastQueryResult. It reports whether the query hit valid geometry together with the hit position, distance, normal, and target path.

  • Ray: Alias of rtx::raytracing::RaycastQueryRay, which defines a ray used as query input.

Query Flow#

Here is the general lifecycle flow of a raycast query:

Create Ray
  ↓
Optional one-time section-plane clipping
  ↓
Submit a copy of the Ray
  ├─ one-shot callback
  ├─ persistent single-ray sequence
  └─ persistent ray-array sequence
  ↓
Wait for a rendered update
  ↓
RTX batches pending rays using the current renderer scene/view state
  ↓
GPU traces each ray
  ↓
Result becomes available
  ├─ completed hit:  valid == True
  └─ completed miss: valid == False
  ↓
Callback runs, or sequence caller polls the latest completed value

0. Acquire a raycast query interface:#

The raycast query interface is the broker between you and RTX for handling raycasts and polling for results. You can just make one and hold onto it for as long as you need.

import omni.kit.raycast.query

raycast = omni.kit.raycast.query.acquire_raycast_query_interface()

1. Create a Ray#

Rays contain a simple origin and direction, with optional arguments for minimum/maximum distance and whether to adjust the ray start/end based on section planes. (For more on section planes, see Section Tool and RTX Section Plane settings).

from omni.kit.raycast.query import Ray

origin = (start_x, start_y, start_z)
direction = (dir_x, dir_y, dir_z)

ray = Ray(
  origin,
  direction,
  min_t=0.0,
  max_t=float("inf"),
  adjust_for_section=True
)

This will cast a ray along the direction vector, starting min_t away from the origin point and continuing up until max_t from origin.

IMPORTANT:

Point calculation is based on the general formula:

point = origin + t * direction

This means that min_t and max_t are not world or stage units, but rather multipiers along direction. If direction is a unit vector, then min_t and max_t will happen to match world distance. But if direction is a non-unit vector like (1, 1, 0), then you will end up with cases where t = 1.0 becomes a distance of sqrt(2.0).

If you want t and its related parameters to be in world units, you have to make sure that direction is normalized when constructing each ray.

2a. Query Single Ray#

You can query a single ray directly if you supply a callback method:

from omni.kit.raycast.query import Ray, RayQueryResult

def callback(ray: Ray, result: RayQueryResult):
  ...

raycast.submit_raycast_query(ray, callback)

In this case, the raycast query system will automatically invoke the callback with a copy of the Ray parameters and a RayQueryResult instance. The result of submit_raycast_query(...) is always None, even when unable to initiate the query.

2b. Set Up a Raycast Sequence#

Raycast sequences are persistent collections of 1-or-more Ray objects which are requeued and queried on every Kit update (usually once per frame).

Raycase sequences require you use a polling method. They can be especially helpful for when you need to cast multiple rays, or the same rays across multiple frames, and are generally better optimized than many single queries with callbacks. This requires first creating a raycast sequence, which you track using a simple ID.

seq_id = raycast.add_raycast_sequence()

# For single ray
result = raycast.submit_ray_to_raycast_sequence(seq_id, ray)

# For multiple rays at once
result = raycast.submit_ray_to_raycast_sequence_array(seq_id, ray_array)

seq_id == -1 indicates that something went wrong and the system was unable to create a raycast sequence.

result from submitting a query will be one of:

  • SUCCESS: the ray (or collection of rays) was/wer accepted for asynchronous processing

  • RAYCAST_SEQUENCE_DOES_NOT_EXIST: you supplied an invalid ID for seq_id

  • RAYCAST_QUERY_MANAGER_DOES_NOT_EXIT: RTX raycast query is unavailable

  • INVALID_PARAMETER: the sequence mode or array size is incompatible (e.g., if you submit multiple rays, then the single submit_ray_to_raycast_sequence() will return INVALID_PARAMETER because the expected size is larger than 1; you could call submit_ray_to_raycast_sequence_array() with a single-element array to reset the counter to 1 and fix the error in this case).

Do not submit an empty array using submit_ray_to_raycast_sequence_array().

NOTE: Submitting a ray (or collection of rays) overwrites all existing rays in the sequence.

3. Get Results#

Because they run as part of the RTX pipeline, raycasts are asynchronous. You will never receive a completed query result without waiting at least one frame, and the delay can sometimes be affected by GPU load and require a longer wait. (Usually, you’ll see results in the next frame, but there’s no guarantee.)

For single raycasts submitted with a callback, the callback will occur on its own. Assuming the result argument is valid, this callback is invoked on Kit’s main/update thread shortly before Hydra rendering. If no collection is available, however, the callback can be invoked synchronously before submit_raycast_query() returns; however it is an error to submit with an invalid sequence ID.

Keep callbacks short to avoid delaying the renderer. The safest practics is to copy any important results to be processed later, during normal extension update code.

If using a raycast sequence ID, you poll results from your own extension code once per frame:

# For single ray result only
status, seq_ray, result = (
  raycast.get_latest_result_from_raycast_sequence(seq_id)
)

# For all ray results
status, seq_rays, results = (
    raycast.get_latest_result_from_raycast_sequence_array(seq_id)
)

status will be:

  • RAYCAST_RESULT_NOT_READY: wait another frame and try again

  • SUCCESS: there is a completed batch of ray queries

  • Potentially some other sequence or parameter error

NOTE: status will always remain SUCCESS as soon as there is a complete batch of ray queries. This means that SUCCESS could indicate a previous batch of rays while a newer batch of different rays is currently pending.

On status == SUCCESS, you may iterate over every ray and result in seq_ray/s and result/s to test for ray validity.

If you regularly change the rays in the sequence, you should verify the array properties (origin, direction, etc) to make sure which batch the results represent so you aren’t accidentally treating a stale batch like a new batch.

result.valid == True indicates that the ray hit valid geometry. result.valid == False indicates that no valid geometry was hit.

The rest of the result fields are only valid if result.valid == True:

  • hit_position: the global coordinate of the ray hit

  • hit_t: the distance from ray origin along the direction vector

  • normal: the surface normal of the geometry (triangle) that was hit

  • instance_id: a low-level renderer ID for the hit geometry

  • primitive_id: a slightly higher-level USD face or primitive ID for the hit geometry

  • get_target_usd_path(): a helper method that retrieves the hit geometry’s prim path as a string, or "" for invalid hits.

get_target_usd_path() relies on the current USD context, and is invalid if the stage has closed or been replaced.

4. Cleanup#

If you created a raycast sequence ID and you no longer need to perform raycasts, you should remove the sequence from the raycast query system, otherwise it will continue to query the same ray(s) over and over every frame. Removing the sequence also releases any render-side resources that were being kept alive for it.

raycast.remove_raycast_sequence(seq_id)

All other types are self-contained simple data, and are safe to drop normally.

Performance and Notes#

Rays intersect with both front AND back faces of geometry.

When non-pickable geometry is hit, the ray continues from 1e-3 (0.001) units further along the direction vector to avoid re-hitting the same location, which can cause the ray to miss hitting, e.g., coplanar pickable geometry.

A ray will handle at most 32 hits with non-pickable geometry (including backfaces) before giving up. This means that a ray could only skip 16 (non-coplanar-edged) boxes along its direction vector before returning result.valid = false (i.e., “no hit detected”).

Closing a stage removes all raycast query sequences, making their IDs invalid. You may need to track stage changes and re-create your query sequence(s), for example, if the user changes the loaded stage.

In general, a few raycasts on their own won’t produce any noticeable delay in the stage rendering. But if you are constantly casting a large number of rays, especially against large complex scenes, or with non-pickable geometry, the time can add up.

Short rays tend to be executed faster as they have less possible interactions to consider with large scenes.

For raycast API examples, see Python usage.

Pickability and filtering#

Raycast Query honors the USD context’s transient pickability state set through omni.usd.UsdContext.set_pickable(). Marking a prim non-pickable leaves it visible in rendered viewports, but Raycast Query attempts to continue behind it to find the next pickable hit. The call walks the supplied USD subtree and updates the renderable descendants that exist at call time.

Pickability is shared renderer state, not a filter local to one query, and has important traversal and lifetime limitations.

Changing pickability does not author an inherited USD property or install a persistent subtree rule.

Pickability is shared renderer state, not a per-query filter, so the geometry is also ignored by viewport (i.e., mouse) picking and any Raycast Query users. It does not disable physics collision.

Important limitations of this technique:

  • Non-pickable geometry is still encountered by the ray, just ignored. Only up to 32 trace iterations are handled by the ray shader before it reports no hit. A closed mesh consumes more than one iteration because entering and exiting surfaces may both be hit.

  • When the ray skips a non-pickable surface, it is offset by 1e-3 to avoid hitting the same surface. This can cause it to miss a pickable coplanar polygon. Avoid placing a required hit surface at the same depth as excluded geometry.

  • The pickable assignment subtree walk only includes current geometry paths that exist when set_pickable() is called. New geometry under the path does not inherit the pickability state. Reapply it after adding renderable descendants at new paths.

  • Pickability state is retained for exact renderable paths, so recreating geometry at the same leaf path preserves its non-pickable state.

  • Restore pickability before deleting or restructuring the excluded subtree. Otherwise retained leaf-path state can affect geometry created later at an identical path.

  • Work scales with the number of renderable descendants and active Hydra engines. Walking and updating a large or deeply instanced subtree can cause a noticeable delay; repeated reapplication multiplies that cost.