Localizing libraries and plugins from kit

The usdrt repo is currently being used to develop runtime components of Omniverse as standalone set of libraries with no complex interdependencies on kit (no kit interfaces, no kit extensions, etc). Ideally everything in this repo depends on nothing more than carbonite and USD.

However, Fabric is a fundamental requirement of these runtime libraries, and Fabric currently ships within Kit. In order to support independent development of runtime libraries, a copy of Fabric code is maintained here and periodically synced from the Kit repo. Because Fabric is structured as a carbonite plugin, this generally does not cause issues, and the libraries built within USDRT can be used inside Kit with a version of Fabric built with Kit, as long as the ABI version of Fabric matches between the two builds.

The kitsync process

A tool named kitsync has been developed to facilitate the localization process, and staying up-to-date with the latest developments in kit.

./kitsync.(bat/sh) -h

kitsync is a python script with a simple set of functions for syncing code from kit to usdrt. Code is synced from the master branch on the usdrt-extras/kit fork for this purpose (which mirrors kit master). kitsync will clone this repo in a sibling directory to the usdrt repo named kit_usdrt.

You may optionally check out a specific branch in the kit_usdrt clone (instead of using master), and specify the -my_branch flag to use that instead of master.

Sync code from kit to usdrt

To update code in usdrt from kit, use the kitsync command with no flags:

./kitsync.bat

The files and directories that will be synced are maintained in this registry:

tools/kitsync/master.json

This is a simple dictionary of files and directories in kit, and their corresponding paths where they should live in usdrt.

If new directories are added to the registry, then also update the repo_format section of usdrt’s repo.toml (in the root of the repository) to exclude the new files from formatting (kit code isn’t subjected to regular formatting and it will be too complicated to compare with kit if we run formatting on it in the usdrt repo).

Commit the changes to the registry and repo.toml (kitsync will bail if the repo is dirty to avoid accidental overwrites - or use the -f flag to ignore repo state and sync anyway).

To skip committing the new files (if you want to check results first), use the -skip_commit flag:

./kitsync.bat -skip_commit

If these files require modification to build in the usdrt repo (removing kit-specific APIs), make and commit those changes in the kit repo and file an MR to add the changes back to kit.

Once those changes are committed and kit builds and passes tests, they can by synced back to usdrt:

./kitsync.bat

Solving the Chicken and Egg problem

Fabric is developed entirely out of the Kit repo. And USDRT has a dependency on Fabric that we are solving by making a local copy of Fabric code. Kit also uses the USDRT package and the libraries it contains. So we have a bit of a circular dependency:

This is obviously undesirable but workable until Fabric moves into its own repository / package, at which point the dependency structure becomes acyclic:

But in the meantime we have a puzzle to solve. When Fabric in Kit changes the plugin version of its carbonite plugins (or introduces an otherwise ABI-breaking change), then the Fabric in Kit is no longer compatible with the copy of Fabric that was used to build USDRT. This causes the USDRT plugins to fail at runtime in Kit, as carbonite will detect that they were built with an incompatible version of Fabric plugins:

So, when Fabric introduces a version or ABI-breaking change in Kit, we need a new build of USDRT that uses the updated Fabric code. That build of USDRT must be included as part of the MR that introduces the breaking Fabric change. Here’s the steps to follow to accomplish that task.

Make a new branch in the usdrt repo off of the develop branch

Give it a name following the usdrt repo conventions (see CONTRIBUTING.md for more detail). For the purpose of this example, we’ll call the branch dev/OM-12345.

(One time) Set up kitsync

If this is your first time going through the process, run kitsync in setup mode to set up the sync repo. This will clone kit from the usdrt-extras fork in a directory adjacent to your usdrt clone, named kit_usdrt

./kitsync.bat setup

Check out your branch in the kit_usdrt clone

Add the repo where your branch lives as a remote in the kit_usdrt clone, and then checkout that branch.

git remote add ablevins ssh://git@gitlab-master.nvidia.com:12051/ablevins/kit.git
git fetch ablevins
git checkout OM-12345

Use kitsync to sync Fabric changes to usdrt

In the usdrt repo, use the kitsync tool to sync your Fabric changes to your usdrt branch.

./kitsync.bat -my_branch

Build and test usdrt

In the usdrt repo, build and test your Fabric changes:

./build.bat -r
./repo.bat test
./repo.bat test -s python

In most cases, the build will succeed immediately. Sometimes, usdrt code will need updates - make those changes now.

If Fabric code needs to be updated, make those changes on your kit branch and re-run kitsync.

Push your usdrt branch and start a new MR

git push origin dev/OM-12345

Open a new MR in Gitlab, targeting the develop branch.

Every time a merge request is created or updated in USDRT, TeamCity USDRT Deploy subproject will do a new build and publish an MR release to packman. We can use that package to validate the sync process in Kit.

Test Kit with the MR-build of usdrt

In your original kit branch, update the usdrt packman dependency to use the new MR package that includes the Fabric changes. In the file:

kit\deps\target-deps.packman.xml

The usdrt entry would change from something like this:

  <dependency name="usdrt" linkPath="../_build/target-deps/usdrt">
      <package name="usdrt" version="3.0.1+deploy.422.0cd39900.teamcity.${platform}" platforms="windows-x86_64 linux-x86_64 linux-aarch64"/>
  </dependency>

To something like this:

  <dependency name="usdrt" linkPath="../_build/target-deps/usdrt">
      <package name="usdrt" version="3.0.1+Merge-Request.493.b2784b65.tc.${platform}" platforms="windows-x86_64 linux-x86_64 linux-aarch64"/>
  </dependency>

At this point, you can rebuild kit. If the USDRT scenegraph tests pass, then the sync was successful:

./_build/windows-x86_64/release/tests-usdrt.scenegraph.bat

If they fail, then either:

1. The tests changed in the usdrt repo, and need to be updated in kit (I usually just copy over the new tests)

2. The sync was not successful - use a diffing tool to compare code between your kit_usdrt repo and the usdrt repo, and resolve unexpected differences in the usdrt repo manually.

Deploy USDRT and switch Kit branch to deployed package

When you are ready to ship your Kit MR with breaking Fabric changes, it should be updated to use a deployed USDRT package instead of an MR package. To do this:

• Note on the usdrt MR that your changes are ready to merge, and ping Alan or usdrt team member

• The MR is merged into the develop branch of usdrt

• Alan or usdrt team member updates usdrt version if necessary, and merges develop into deploy

• Teamcity builds an official package every time the deploy branch is updated

• Update kit\deps\target-deps.packman.xml with the deployed usdrt package version in your kit branch

Merge your kit change

At this point, your kit branch includes your Fabric changes, it uses a new build of usdrt, and it passes all tests on TeamCity. Merge it into kit integ-master whenever you are ready.

Log of localization

Sync changes from kit 03/20/23

Sync Fabric changes up to kit commit d02f15d4baf920fc52063610db22f2773bd94c4b

Sync changes from kit 03/17/23

Sync Fabric changes up to kit commit 531ebfdc6c60d0b9c384527c017c0faa023d96ba including StageAtTimeInterval wrapper changes

Sync changes from kit 03/10/23

Sync Fabric changes up to kit commit 215fbd41c96d88df3885a70400e84d179a42c055 including mgpu changes and also crash fix from OM-85811

Sync changes from kit 02/28/23

Sync Fabric changes up to kit commit 701b36eeb536673c72f09eb75229dad0406a3c17

Sync changes from kit 02/24/23

Sync Fabric including critical FSD fix from OM-84015

Sync changes from kit 01/26/23

Sync Fabric to include new minimal populate functionality at kit commit 3e3aa6a86b48314ab5360afbdedab0a1f9a51ace

Sync changes from kit 01/12/23

Sync Fabric to include new hierarchy functionality

Sync changes from kit 12/9/22

First sync following Fabric rebrand

Sync changes from kit 9/19/22

Synced changes from Kit for MR for OM-35065 to add mutli-rel support to Fabric. https://gitlab-master.nvidia.com/omniverse/kit/-/merge_requests/19058

Sync changes from kit 7/27/22

Synced changes from Kit MR for OM-56223 to resolve ABI change. https://gitlab-master.nvidia.com/omniverse/kit/-/merge_requests/18071

Sync changes from kit 7/12/22

Updated with latest changes from kit at kit commit 8c0842aa4ee146d3898c4574d2119ce457599265

Sync changes from kit 6/28/22

Updated with latest changes from kit at kit commit 8a144d556e2439e5e547fdcbce296f17fdd8da60

Sync changes from kit 6/15/22

Updated with latest changes from kit at kit commit 879aa42840ed6054299b8570781801252b27eb4a

Includes move flatcache move to rendering and Dan’s mega refactor

Sync changes from kit 6/7/22

Updated with latest changes from kit at kit commit 3b516490f4a1bc8a495aa9a536f5d8211f7d3d9c

Removed omni.inspect experiment

Sync changes from kit 5/24/22

Updated with latest changes from kit at kit commit d6a7e501a7ad4da97aa4b80b6fb8f85dce96d5f1

Added premake utils to support compiling .cu files

Sync changes from kit 5/2/22

Updated with latest changes from kit at kit commit 51598aa4270fd510d029bb029094168a11894837

Add support for syncing usdrt::Gf 4/15/22

usdrt::Gf will soon be a delivered with kit in addition to usdrt, so a new -branch flag has been added to allow syncing with a Gf-specific branch. Ongoing development on usdrt::Gf should still continue in the usdrt repo and this gives us a simple way to make MRs back to the kit repo.

Sync omni::math::linalg change back to kit 4/12/22

The final push to complete usdrt::Gf required a number of changes to linalg, and all of those are synced here.

Also added the unit test for omni.inspect, which was missed previously in the kitsync.

Changes merged back to usdrt-extras kit at kit commit ea2551d7de71a2e473649b030360cf26313ea67e

Sync from kit 4/4/22

Rebased usdrt-extras/kit onto kit master at 08836d5b16f5b67d9b56e4b2c7c42d67b2d192eb and synced changes from kit

Sync omni.inspector changes back to kit 4/4/22

Kevin Picott isolated omni.inspector to build and run within usdrt repo.

Changes merged back to usdrt-extras kit at kit commit eee6c24e11b4759bb5457b1751f842d7d997891c

Sync omni::math::linalg change back to kit 3/25/22

OM-45720 added buffer protocol support Gf classes, which in turn required exposing the data member of the quat class.

Changes merged back to usdrt-extras kit at kit commit 21396a08db41da609b9675f63201d39f6e0fdf85

Sync omni::math::linalg changes back to kit 3/17/22

OM-46071 required minor changes to omni::math::linalg in order to build Python bindings on linux: https://gitlab-master.nvidia.com/omniverse/usdrt/-/merge_requests/57

These changes were merged back to usdrt-extras/kit at kit commit 9a2fe72bc62856e7ec68c63d363bdd2460669556

Sync from kit 3/11/22

Rebased usdrt_kit_sync branch of usdrt-extras/kit to kit commit 337d6652971d654df686583482d1d45120ca7766

Synced code from usdrt-extras/kit to usdrt using kit commit ee66c76ff07e1df33ede06e2c314fef74af94ceb

Updated packman dependencies to match kit (nv-usd, carb-sdk-plugins, cuda)

Updated code to build in usdrt and synced back to usdrt-extras/kit at kit commit 25ce6bca69bb16447c222e89ad11d2194dc887be

kitsync tool added 2/22/22

In order to facilitate updates from kit, developed and added the kitsync tool (details at the top of this file).

Updated repo_format to prevent formatting from running on RT code copied from kit.

Re-sycned from kit commit c20d497e34401e0b68547f703c7f7e2b166f3df6 and re-wrote usdrt-required changes.

Synced back to kit at kit commit cb3e99fd8e672a67bd0b3dfe73ed589f254f3cde and pushed to usdrt-extras/kit

omni::math::linalg 2/16/22

Copied linalg library (include/omni/math/linalg) form kit at commit c20d497e34401e0b68547f703c7f7e2b166f3df6

This is used as the basis for usdrt’s Gf implementation.

Modifications

Removed SafeCast.h, but it’s probably okay to add it later if it becomes necessary.

Ran repo_format, which modified formatting on some of this code.

flatcache and omni.gpucompute-cuda 2/14/22

Switched from using a kit submodule to a copy of the code from kit at commit c20d497e34401e0b68547f703c7f7e2b166f3df6

Targeted libraries

The stuff I copied because I want to use it:

• flatcache

  • include/omni/fabric

  • source/plugins/carb.flatcache

• omni.gpucompute-cuda plugin

  • include/omni/gpucompute/GpuCompute.h

  • source/plugins/omni.gpucompute-cuda

Dependencies

The other stuff I had to copy to make the above work:

• Omnigraph types (used by flatcache)

  • include/omni/graph/core/Type.h

• Realm includes

  • include/gpu/realm/IRealm.h (required by Fabric.cpp)

  • include/gpu/realm/*.h (required IRealm.h)

• Other includes needed to satisfy Realm

  • include/carb/cuda

  • include/carb/cudainterop

  • include/carb/glinterop

  • include/rtx/resourcemanager

  • include/rtx/rtx (needed by rtx/resourcemanager)

  • include/gpu/foundation

  • include/carb/graphics (needed by gpu/foundation)

Modifications

I removed the TestPathToAttributesMapD3dVk.cpp flatcache unit test to avoid the need to build omni.gpucompute-d3dvk

Includes to Fabric.cpp were modified slightly to avoid an import of omni/kit/IApp.h

Ran repo_format, which modified formatting on some of this code.