Object Handling

Objects in a facility change carriers over time: parts on robots, pallets on automated guided vehicles (AGVs), bottles on conveyors. This guide describes how to represent those carrier changes in OpenUSD (Universal Scene Description), what each representation costs, and how to evaluate them at runtime.

Sample Implementation

Repository location: VFI Object Handling Scripts (GitHub)

The samples generate a scene, evaluate it, and drive it in the viewport. The sample scene has 1,000 instances that ride one animated carrier, transfer one at a time to a second carrier, and ride the second carrier away, keeping their prim paths throughout.

Component	Purpose
create_carrier_scene.py	Generates the scene, in carrier and baked form
compute_carrier_transforms.py	Evaluates the arrays offline, bakes them, or verifies them
set_instance_carrier.py	Attaches instances to a different carrier
omni.vfi.object_handling	Extension driving the instancer from the timeline

Generate a stage with create_carrier_scene.py, then open it in an Omniverse Kit app with the extension enabled. The stage has its own camera at /Factory/DemoCamera. Click Play.

The console reports the instance count and the result of reading the positions and orientations back out of Fabric, the runtime data store the renderer reads. A successful read-back confirms that the extension is driving the stage.

Problem Statement

The prim hierarchy is the only mechanism for spatial parenting in OpenUSD. When an ancestor moves, its descendants move with it. A prim’s path, and therefore its parent, is the same at every time code. That stability is what makes caching, composition, and deterministic playback work.

As a result, a prim cannot change parents over time. You cannot author “this bottle is parented to the robot through frame 100 and to the AGV from frame 101”.

UsdNamespaceEditor and relocates can reparent a prim, but they are authoring operations. They are not animatable and cannot be triggered at runtime.

Two workarounds are common: duplicating the object under every carrier and toggling visibility, and baking world-space transforms at every time sample. Neither scales to facility size, and visibility toggling is the one to avoid outright, for the reasons under Approaches to Avoid. The carrier pattern stores neither.

Solution

Represent transported objects as instances of a UsdGeomPointInstancer and record which carrier each instance is on, rather than where it is.

That takes two pieces, and the rest of this guide follows the split.

1. Authoring / OpenUSD. Custom attributes on the instancer, naming the carriers, which carrier each instance is attached to at each time code, and each instance’s offset and orientation in that carrier’s space. They are custom attributes rather than part of an OpenUSD schema, and the renderer does not read them.

2. Runtime / Fabric. A Kit extension that reads those custom attributes each frame, composes each instance’s world position and orientation from its offset and the transform of the carrier it is on, and writes the result into Fabric as positions, orientationsf and orientations. Those are the attributes the renderer reads.

No world position is stored in the stage. The extension computes one every frame.

1. Authoring / OpenUSD

Transported objects are instances of a UsdGeomPointInstancer, the representation that scales to facility size. The instancer holds four properties:

1. A relationship naming every carrier the instances can attach to. Its targets are static, so switching carriers does not restructure the stage.

2. A time-sampled index array selecting the active carrier for each instance. This is what makes attachment animatable.

3. A local offset array placing each instance in its carrier’s space.

4. A local orientation array giving each instance’s facing in that space.

The three arrays are indexed the same way as protoIndices and positions.

An instance’s index ties the arrays together. The same element of every array describes one instance. Where an index is never reused, it also serves as the instance’s identity: map the identifiers your source system uses onto indices once, and that mapping holds for the life of the stage. Where indices are reused, author ids, which is described under Parts Entering and Leaving.

def PointInstancer "Parts"
{
    custom rel      carriers      = [</Factory/Robot>, </Factory/AGV>]
    custom token    ownershipMode = "carrier"
    custom int[]    instanceCarriers.timeSamples     = { 1: [0, 0, ...], 101: [1, 1, ...] }
    custom float3[] instanceOffsets.timeSamples      = { 1: [...], 100: [...], 101: [...] }
    custom quatf[]  instanceOrientations.timeSamples = { 1: [...] }

    point3f[] positions     = [...]   # schema attribute; written at runtime
    quatf[]   orientationsf = [...]   # schema attribute; written at runtime
    quath[]   orientations  = [...]   # schema attribute; written at runtime
}

At runtime, compose each instance’s offset and orientation with the world transform of its active carrier. That produces the positions and orientation values, which the runtime writes into Fabric, where the renderer reads them.

Precision of orientations

orientations, the default orientation array of UsdGeomPointInstancer, is half precision and cannot hold an orientation exactly. Instances render slightly rotated from where they were placed, and orientations interpolated between authored samples jitter.

orientationsf holds the same orientations at full precision. No Kit release renders from it yet, and a future release will. Author both arrays: the runtime fills them from one result, so the stage renders from orientations today and gains the precision when that release arrives, with no change to the stage.

Author placeholder values for positions, orientationsf, and orientations even though the runtime overwrites them. For the reason, see Runtime / Fabric.

Instances cannot be selected or overridden the way a prim can, and they cannot take part in physics, which needs a rigid body prim for each object.

Local Offsets

Author offsets in carrier space, not world space, for three reasons:

Source data is already local.
Applications that reparent internally hold local matrices. The composed world matrix is an internal render-time product, so asking for world transforms demands data that integrators do not have.

World space multiplies animated data.
Consider bottles in a crate on a pallet carried by a forklift. With local transforms, animating the forklift moves everything below it. In world space, every bottle needs its own animated transform.

Array attributes are dense.
Moving one instance rewrites time samples for the whole array. Sampling offsets only at handoff events limits how often the array is rewritten.

Using local offsets affects only what is stored. What is computed at runtime is still world transforms, because renderers consume world matrices.

Handoff Continuity

Author instanceOffsets and instanceOrientations at both the last time code before the carrier changes and the first time code after it. Float and quaternion arrays both interpolate between time samples when the bracketing samples are the same length, so a single sample makes instances slide and turn between carriers across the handoff.

Integer arrays are held rather than interpolated, so instanceCarriers needs only one sample per change. Author the incoming carrier’s index at the time code the handoff takes effect.

Ownership Modes

ownershipMode marks which instancers the carrier evaluation drives. Set it to carrier. An evaluator skips any instancer with another value, so one stage can hold carrier-driven instancers alongside instancers driven by another mechanism.

Carrier Assignments

To assign an instance to a carrier, author the carrier’s index at that instance’s position in instanceCarriers. The array is time-sampled, and each sample holds one index per instance: the carrier that instance is attached to at that time code.

The array is not sparse. A sample describes the assignment for every instance, so adding or updating a sample means writing the whole array, with the correct index for every instance. It is not a record of what changed, and there is no way to author “instance 5 moved to the AGV at frame 50” on its own.

A sample then governs playback until the next authored sample replaces it. Any sample later in the animation was authored before this change and still holds the instance’s old index. Playback reaching it moves the instance back onto its previous carrier, mid-animation and without an error. Author the change into every later sample as well.

The same applies when a part leaves and a new one takes its index: later samples still hold the previous part’s carrier at that position. Parts Entering and Leaving covers reusing indices.

from pxr import Usd, Vt

INSTANCE_CARRIERS_ATTR = "instanceCarriers"


def set_instance_carrier(stage, instancer_path, instances, carrier, frame):
    """Attach instances to a carrier from frame onward."""
    attr = stage.GetPrimAtPath(instancer_path).GetAttribute(INSTANCE_CARRIERS_ATTR)

    # The requested frame itself, so the change takes effect there rather
    # than at the next sample that happens to exist, plus every later
    # sample, so none of them undoes it.
    times = sorted({float(frame)} | {t for t in attr.GetTimeSamples() if t >= frame})

    for time_code in times:
        assignment = list(attr.Get(Usd.TimeCode(time_code)))
        for index in instances:
            assignment[index] = carrier
        attr.Set(Vt.IntArray(assignment), Usd.TimeCode(time_code))

Attaching instances 500, 501, and 502 to carrier 1 from frame 50 authors a sample at 50 and rewrites every sample after it, and those instances follow carrier 1 for the rest of the animation. set_instance_carrier.py runs the same operation from the command line.

Parts Entering and Leaving

The instance arrays are parallel and fixed in length, so a facility where parts appear and disappear cannot grow and shrink them freely. Two constraints apply:

• Changing the length means re-authoring every array at that time code, including protoIndices.

• Arrays interpolate between time samples only when the bracketing samples are the same length. A length change turns interpolation into held values for that interval, so offsets snap instead of sliding.

Allocate the instancer once at the highest concurrent part count and keep the length fixed. A part entering the facility takes a free index; a part leaving returns one.

Reusing indices separates an instance’s identity from its position in the arrays, so author ids. The ids attribute is the instancer’s identity attribute: an int64 per instance, time-sampled where identities change. Each value stays with the part as it enters, rides, and leaves. Give each part a distinct id, and reuse indices as needed.

Hide the instances that are not live by listing their ids in invisibleIds. Unlike visibility toggling on duplicated prims, this hides instances inside the one instancer, so the prim count does not change. Changing one instance’s visibility means re-authoring every time sample of that attribute, so budget for it the way you budget for the carrier assignment array.

Choosing the allocated count is the design decision that the fixed length forces. Storage and per-frame cost both scale with that count, not the live count.

Transport Data from External Simulations

Transport data arriving from an external simulation follows the two rules in Carrier Assignments, with three additions:

• Map the source system’s attachment to the carriers relationship. The source system already tracks which resource (the carrier) holds each item, which maps directly onto the relationship and the active index.

• Author on events, not on frames. Author the carrier index and local offset when a handoff happens. Sampling per frame brings back the storage cost of baked transforms.

• Keep local matrices local. Author the offset in carrier space instead of composing to world space before export.

2. Runtime / Fabric

Authoring and evaluation produce a correct stage. Rendering it takes one more step.

The renderer reads Fabric, not USD. Authoring positions into USD every frame is slow, and the values do not reach the renderer directly. Compute the arrays, then write them into Fabric.

Write Persistence

A carrier-driven instancer holds static USD data. Its positions and orientations come from the carrier’s transform, not from authored values, so no USD attribute changes between frames.

The Fabric Scene Delegate re-populates a prim only when its USD data changed since the last frame. A carrier-driven prim does not change during playback, so the delegate leaves it alone and the values written each frame survive to render time.

Evaluating One Frame

Discover the instancers by traversing USD, and subscribe to the application update stream unconditionally. Then, for each instancer:

import numpy as np
import usdrt
from pxr import UsdGeom


def quat_multiply(a, b):
    # Hamilton product of two (N, 4) arrays of (w, x, y, z) quaternions.
    aw, ax, ay, az = a[:, 0], a[:, 1], a[:, 2], a[:, 3]
    bw, bx, by, bz = b[:, 0], b[:, 1], b[:, 2], b[:, 3]
    return np.stack([
        aw * bw - ax * bx - ay * by - az * bz,
        aw * bx + ax * bw + ay * bz - az * by,
        aw * by - ax * bz + ay * bw + az * bx,
        aw * bz + ax * by - ay * bx + az * bw,
    ], axis=1)


def write_frame(stage, rt_stage, instancer, frame):
    prim = instancer.GetPrim()
    indices = np.array(prim.GetAttribute("instanceCarriers").Get(frame), dtype=np.int32)
    offsets = np.array(prim.GetAttribute("instanceOffsets").Get(frame), dtype=np.float32)
    base = np.array(prim.GetAttribute("instanceOrientations").Get(frame), dtype=np.float32)

    # Each carrier's transform in the instancer's space, in relationship
    # order. positions and orientations are in the instancer's own space and
    # the renderer applies the instancer's local-to-world on top, so a
    # transform on a shared ancestor lands twice unless it is divided out
    # here.
    cache = UsdGeom.XformCache(frame)
    to_instancer = cache.GetLocalToWorldTransform(prim).GetInverse()
    carriers = [cache.GetLocalToWorldTransform(stage.GetPrimAtPath(target)) * to_instancer
                for target in prim.GetRelationship("carriers").GetTargets()]

    mats = np.array([[[m[r][c] for c in range(4)] for r in range(4)]
                     for m in carriers], dtype=np.float32)
    rotations, translations = mats[:, :3, :3], mats[:, 3, :3]

    # USD composes row-vector first, so p' = p * R + t. Composing only the
    # carrier's translation leaves instances behind when a carrier rotates.
    positions = (np.einsum("ni,nij->nj", offsets, rotations[indices])
                 + translations[indices])

    # The NumPy view of a quaternion array is (i, j, k, real) while the math
    # here is (w, x, y, z), so both boundaries reorder. The carrier's rotation
    # is applied after the instance's own.
    carrier_q = np.array([[q.GetReal(), *q.GetImaginary()]
                          for q in (m.ExtractRotationQuat() for m in carriers)],
                         dtype=np.float32)
    orientations = quat_multiply(carrier_q[indices], base[:, [3, 0, 1, 2]])

    rt_prim = rt_stage.GetPrimAtPath(str(prim.GetPath()))
    rt_prim.GetAttribute("positions").Set(
        usdrt.Vt.Vec3fArray(np.ascontiguousarray(positions, dtype=np.float32)))
    xyzw = orientations[:, [1, 2, 3, 0]]
    rt_prim.GetAttribute("orientationsf").Set(
        usdrt.Vt.QuatfArray(np.ascontiguousarray(xyzw, dtype=np.float32)))
    # The array every current release renders. orientationsf is written
    # alongside it for releases that begin reading it.
    rt_prim.GetAttribute("orientations").Set(
        usdrt.Vt.QuathArray(np.ascontiguousarray(xyzw, dtype=np.float16)))

Two properties of the stage are required:

Keep the carrier’s transform rigid.
Offsets compose with the carrier’s whole transform, so a scale on the carrier scales the instances with it. Put visual geometry that needs scaling on a child prim, below the carrier. The instancer’s own transform is divided out of the result, so keep the instancer’s transform rigid too.

Author the position and orientation attributes in USD.
Fabric mirrors the attributes the stage has. An attribute absent from USD is absent from the Fabric prim, and the write is a no-op. Placeholder values are enough; the runtime overwrites them.

Verifying the Write

Write once, read positions and orientationsf back out of Fabric, and compare them against what was written. A read-back is a few lines of code, and it shows that the values reached Fabric.

Approaches to Avoid

Two representations express carrier-attached objects without computing anything at runtime. Neither scales to facility size.

Visibility toggling
Add the object under every carrier it can attach to, as a reference or an instance, and toggle visibility at each handoff. Avoid this approach. It increases both storage and runtime cost:

• Prim count multiplies by carrier count, which raises the complexity of the whole stage.

• Hidden prims are still animated and evaluated, so the work happens whether or not the object is visible.

• The visibility change itself is costly. On a deep subtree, it is slow enough to be visible as geometry loading.

• Identity splits across the duplicates, so the object is no longer a single persistent object.

Authoring those duplicates as copies, not as references or instances, compounds every one of those costs.

Baked world-space transforms
Author absolute transforms at every time sample. Playback computes nothing, so reading them back is fast. The carrier relationship is discarded, so any edit to a carrier’s motion means re-baking, and file size grows with object count multiplied by frame count.

Performance

Array Operations

Compute positions and orientationsf with array operations, and convert the results to the runtime array types by passing their buffers directly to the array constructors. Neither step should iterate instances in Python.

Measured over 200 frames, streaming positions and both orientation arrays to Fabric:

Instances	Processor	Compute (ms)	Write (ms)	Total (ms)
1,000	CPU	0.488	0.155	0.642
10,000	CPU	1.199	0.423	1.622
100,000	CPU	7.309	2.730	10.039
100,000	GPU	3.222	2.768	5.990

Per-instance cost falls as the count rises, because reading the carrier transforms costs the same at any instance count. Reading the instance arrays and the carrier transforms is the larger half of the total. The arithmetic and the write are both small.

A GPU kernel is worth using above roughly 50,000 instances. Below that, the host-to-device copy costs more than the arithmetic it replaces, and the vectorized CPU path is faster.

The conversion is where an implementation most easily loses that speed. Building the runtime array element by element costs 3.8 ms at 1,000 instances, compared with 0.004 ms for passing the buffer to its constructor.

These are the costs of computing the arrays and writing them. Frame time also includes rendering the instances and everything else the application does, and is not measured here.

Storage

Two carriers over 200 frames, each instance transferring at its own time. The baked stage holds every instance position and orientation at every frame:

Instances	Carrier pattern	Baked	Ratio
1,000	690 KB	9.2 MB	13.7 times smaller
10,000	6.5 MB	91.5 MB	14.1 times smaller

The three representations scale differently:

Representation	Storage scales with
Visibility toggling	objects × carriers × frames
Baked	objects × frames
Carrier pattern	objects × distinct handoff times

Because array attributes are dense (see Local Offsets), authoring instanceCarriers at a time code stores every element, even when one instance changed. Storage scales with the number of distinct handoff times, not the number of handoffs.

The figures above are the worst case: every instance transfers at its own time, so the attribute holds a time sample at all 200 frames and the only saving is the width of an int against a point3f and a quatf. Lengthening the animation grows both stages together. Where handoffs cluster onto a few time codes, storage stops tracking the frame count and the gap widens.

Scaling

Runtime cost grows with instance count, sub-linearly at the sizes measured, and is independent of animation length. Storage cost is linear in instances multiplied by distinct handoff times. Neither depends on carrier count.

Limitations

Scale
This was tested with up to 10,000 instances.

Relationships are not time-sampled
Relationship targets cannot vary over time, which is why the pattern pairs static targets with an animated index.

Multi-carrier attachment
This guide does not cover an instance attached to two carriers at once.

Instance addressability
The pattern preserves per-instance identity in the data. Selecting an instance in the Viewport still requires editor tooling.