Plugin Creation

This is a practitioner’s guide to using the Execution Framework. Before continuing, it is recommended you first review the Execution Framework Overview along with basic topics such as Graphs Concepts, Pass Concepts, and Execution Concepts.

The Execution Framework is a graph of graphs. EF allows users, with their own code, to:

  • Build the graph

  • Optimize the graph

  • Defines how/when nodes in the graph are executed

  • Provide chunks of code to execute in the graph

  • Customize how graph data is stored

  • Define custom schedulers to dispatch the graph’s tasks

The primary method used to extend EF’s functionality is to subclass from EF’s implementations of its core interfaces: Node, NodeDef, NodeGraphDef, ExecutionContext, Executor, PopulatePass, PartitionPass, etc.

A reasonable questions is, “How are these custom user implementations instantiated by EF?” In short:

Visually:

flowchart TD Application --> PassPipeline Application --> ExecutionContext PassPipeline --> Pass Pass --> NodeDef Pass --> NodeGraphDef NodeDef --> OpaqueCode[Opaque Code] NodeGraphDef --> Node NodeGraphDef --> Executor

Figure 17 Who instantiates who in EF. Upstream entities instantiate downstream entities.

Above, we see there are two objects the application will instantiate: PassPipeline and ExecutionContext. The implementations instantiated here will be application specific.

The creation of all other entities can be tied back to passes. As mentioned above, passes are instantiated by the application’s PassPipeline, which accesses a global registry of available passes. This global registry, available via the global getPassRegistry() function, can be populated by user plugins.

In this article, we do not cover application level customization, such as PassPipeline and ExecutionContext, since such customizations are rare when using the Kit SDK (Kit already does this for you). We will cover how users can create their own plugins to define their own passes, and thereby their own nodes, definitions, and executors. Omniverse has two methods to define plugins: Carbonite Plugins and Omniverse Modules.

Creating an Omniverse Module

The minimum needed to implement an Omniverse module can be found in the omni.kit.exec.example-omni extension:

Listing 10 Example of defining an Omniverse Module using the Kit SDK.
#include "OmniExamplePass.h"

#include <omni/core/Omni.h>
#include <omni/core/ModuleInfo.h>
#include <omni/graph/exec/unstable/PassRegistry.h>
#include <omni/kit/exec/core/unstable/Module.h>

// we need the name in a couple of places so we define it once here
#define MODULE_NAME "omni.kit.exec.example-omni.plugin"

// this is required by omniverse modules
OMNI_MODULE_GLOBALS(
    MODULE_NAME,                           // name of the module
    "Example Execution Framework Module"   // description of the module
);

// this registers the OmniExamplePass population pass.  any time a node named "ef.example.greet" is seen, this pass will
// attach a definition to the node that will print out "hi".
//
// this macro can be called from any .cpp file in the DLL, but must be called at global scope.
OMNI_GRAPH_EXEC_REGISTER_POPULATE_PASS(OmniExamplePass, "ef.example.greet");

namespace
{

omni::core::Result onLoad(const omni::core::InterfaceImplementation** out, uint32_t* outCount)
{
    // this method can be used to register default implementations for objects.  for example, omni.kit.exec.core uses
    // this method to register its singletons: IExecutionControllerFactory, IExecutionGraphSettings, ITbbSchedulerState,
    // etc.
    //
    // this function is not used in this example.
    return omni::core::kResultSuccess;
}

// called once the DLL is loaded
void onStarted()
{
    // this macro must be called by any DLL providing EF functionality (e.g. passes).  it will register any passes found
    // in the module with EF.
    OMNI_KIT_EXEC_CORE_ON_MODULE_STARTED(
        MODULE_NAME,
        []() {
            // this optional function is called when any EF module is unloaded. the purpose of this function is to
            // remove references to any objects that may by potentially be unloaded.
        });
}

// tells the framework that this module can be unloaded
bool onCanUnload()
{
    return true;
}

// called when the DLL is about to be unloaded
void onUnload()
{
    // if OMNI_KIT_EXEC_CORE_ON_MODULE_STARTED() is called, this macro must also be called.  it will inform EF that the
    // DLL is about to be unloaded.  additionally this macro will unregister any passes registered by the DLL.
    OMNI_KIT_EXEC_CORE_ON_MODULE_UNLOAD();
}

} // end of anonymous namespace

// main entry point called by the carbonite framework.
OMNI_MODULE_API omni::core::Result omniModuleGetExports(omni::core::ModuleExports* exports)
{
    OMNI_MODULE_SET_EXPORTS(exports);
    OMNI_MODULE_ON_MODULE_LOAD(exports, onLoad);
    OMNI_MODULE_ON_MODULE_STARTED(exports, onStarted);
    OMNI_MODULE_ON_MODULE_CAN_UNLOAD(exports, onCanUnload);
    OMNI_MODULE_ON_MODULE_UNLOAD(exports, onUnload);

    return omni::core::kResultSuccess;
}

Building the DLL is build system dependent, but when using the Kit SDK, the following snippet from source/extensions/omni.kit.exec.example-omni/premake5.lua should do the job:

Listing 11 Example of building an Omniverse Module using the Kit SDK.
-- start the omnigraph/omni.kit.exec.example-omni project..
project_ext(ext, { generate_ext_project=false })
    -- target: omnigraph/omni.kit.exec.example-omni/omni.kit.exec.example-omni.plugin
    --
    -- builds the c++ code
    project_ext_plugin(ext, ext.id..".plugin")
        add_files("/impl", "plugin") -- add plugins directory to files to be built
        exceptionhandling "On" -- api layer is allowed to throw exceptions (abi is not)
        rtti "Off" -- shouldn't be needed since we're using oni

The omni.kit.exec.example-omni extension is a fully functioning extension found at source/extension/omni.kit.exec.example-omni/. It includes much more than what is presented above, for example, how to create tests for your EF extension. It is a suitable starting point for your own EF extension.

Creating a Carbonite Plugin

The minimum needed to implement a Carbonite plugin can be found in the omni.kit.exec.example-carb extension:

Listing 12 Example of defining an Carbonite plugin using the Kit SDK.
#define CARB_EXPORTS // must be defined (folks often forget this)

#include "CarbExamplePass.h"

#include <carb/PluginUtils.h>
#include <omni/graph/exec/unstable/PassRegistry.h>
#include <omni/kit/exec/core/unstable/Module.h>

// we need the name in a couple of places so we define it once here
#define MODULE_NAME "omni.kit.exec.example-carb.plugin"

// CARB_PLUG_IMPL must be called with an interface.  this is an example interface.
//
// if your plugin does not publish any interfaces, consider using Omniverse Modules rather than a Carbonite Plugin.
struct IExampleInterface
{
    CARB_PLUGIN_INTERFACE("omni::graph::exec::example::IExampleInterface", 1, 0)
};

void fillInterface(IExampleInterface& iface)
{
    // used to populate your interface
}

// required. described the plugin to the carbonite framework.
const struct carb::PluginImplDesc kPluginImpl =
{
    MODULE_NAME,
    "Example Execution Framework Plugin",
    "NVIDIA",
    carb::PluginHotReload::eDisabled,
    "dev"
};

// call CARB_PLUGIN_IMPL_DEPS if your plugin has static dependencies.  this plugin does not.
CARB_PLUGIN_IMPL_NO_DEPS();

// required.  describes the carbonite interfaces this plugin provides
CARB_PLUGIN_IMPL(
    kPluginImpl,
    IExampleInterface
    // add any carbonite interfaces here
)

// this registers the CarbExamplePass population pass.  any time a node named "ef.example.greet" is seen, this pass will
// attach a definition to the node that will print out "hi".
//
// this macro can be called from any .cpp file in the DLL, but must be called at global scope.
OMNI_GRAPH_EXEC_REGISTER_POPULATE_PASS(CarbExamplePass, "ef.example.greet");

// called once the DLL is loaded
CARB_EXPORT bool carbOnPluginStartupEx()
{
    // this macro must be called by any DLL providing EF functionality (e.g. passes).  it will register any passes found
    // in the module with EF.
    OMNI_KIT_EXEC_CORE_ON_MODULE_STARTED(
        MODULE_NAME,
        []() {
            // this optional function is called when any EF module is unloaded. the purpose of this function is to
            // remove references to any objects that may by potentially be unloaded.
        });

    return true;
}

// called right before the DLL will be unloaded
CARB_EXPORT void carbOnPluginShutdown()
{
    // if OMNI_KIT_EXEC_CORE_ON_MODULE_STARTED() is called, this macro must also be called.  it will inform EF that the
    // DLL is about to be unloaded.  additionally this macro will unregister any passes registered by the DLL.
    OMNI_KIT_EXEC_CORE_ON_MODULE_UNLOAD();
}

Building the DLL is build system dependent, but when using the Kit SDK, the following snippet from source/extensions/omni.kit.exec.example-carb/premake5.lua should do the job:

Listing 13 Example of building a Carbonite plugin using the Kit SDK.
-- start the omnigraph/omni.kit.exec.example-carb project..
project_ext(ext, { generate_ext_project=false })
    -- target: omnigraph/omni.kit.exec.example-carb/omni.kit.exec.example-carb.plugin
    --
    -- builds the c++ code
    project_ext_plugin(ext, ext.id..".plugin")
        add_files("/impl", "plugin") -- add plugins directory to files to be built
        exceptionhandling "On" -- api layer is allowed to throw exceptions (abi is not)
        rtti "Off" -- shouldn't be needed since we're using oni

Deciding on Which Approach to Take

When implementing new EF functionality, it is recommended to use Omniverse modules. Omniverse modules work well with EF’s ONI based interfaces. Additionally, if you plan on providing your own ONI interfaces that encapsulate global state that needs to be accessed across many DLLs, Omniverse modules allow you to register interfaces via omni::core::ITypeFactory. See omni.kit.exec.core for an example.

If you are extending an existing Carbonite plugin with EF functionality (e.g. omni.graph.core) using the existing Carbonite plugin is the path of least resistance. By taking this approach, your new EF implementation will be able to access implementation details of existing functionality located in the same plugin.

Avoiding Crashes at Exit

Note

This section covers a crash on exit problem often seen when using the Kit SDK. The solution provided is not implemented in the core EF library, rather it is implemented in the omni.kit.exec.core extension, which bridges EF with Kit. Both the problem and solution are presented here, in the core EF docs, to help users of EF outside of the Kit SDK to understand potential edge cases with EF integration.

Applications based on the Kit SDK will shutdown each extension/plugin/module before exit. This can lead to unexpected crashes when DLLs depend upon each other. This coupling of functionality between DLLs is often the case in EF.

As an example, consider the omni.graph.action extension, which provides definitions and passes to implement OmniGraph’s Action Graph extension. The omni.graph.action extension depends upon omni.graph.core which in turn depends upon omni.kit.exec.core, which depends upon the core EF extension (omni.graph.exec). When the application starts, this dependency information is used to load omni.graph.exec first, followed by omni.kit.exec.core second, then omni.graph.core, and finally omni.graph.action. During shutdown, the extensions are unloaded in reverse order.

flowchart LR oge[omni.graph.exec] -- Provides PassRegistry To --> okec[omni.kit.exec.core] okec -- Provides ExecutionController To --> ogc[omni.graph.core] ogc -- Provides OG To --> oga[omni.graph.action] oga -. Stores Data In .-> ogc ogc -. Stores Data In .-> okec

Figure 18 Safely unloading extensions is no easy task. Explicit extension dependencies are depicted with solid lines. Implicit reference counting dependencies are depicted with dotted lines.

During shutdown, omni.graph.action will unload without issue. However, when unloading omni.graph.core you’re likely to see a crash when OmniGraph’s destructs its internal objects. This is because OmniGraph stores an ObjectPtr to each EF definition is creates. This isn’t a bug, as it allows OmniGraph to quickly and precisely invalidate parts of EF’s execution graph. However, during shutdown, definitions provided by omni.graph.action will crash, because attempting to invoke their destructors will call into unloaded code.

EF’s solution to this problem is OMNI_KIT_EXEC_CORE_ON_MODULE_STARTED(). This macro’s second argument is a callback function that will be invoked before any DLL that called OMNI_KIT_EXEC_CORE_ON_MODULE_STARTED() is unloaded. This gives EF enabled DLLs the opportunity to clean up any references to objects implemented in external DLLs that are possibly about to be unloaded.

Next Steps

Above, we covered the creation of plugins to extend EF’s functionality. Readers are encouraged to move onto to either Definition Creation, Pass Creation, and Executor Creation to being implementing new graphs.