Carbonite Framework

The key ingredient of Carbonite is what we call the Carbonite carb::Framework. It is the core entity which manages plugins and their interfaces. This guide describes the plugin management functionality of the Carbonite Framework. The Framework itself is versioned and when a plugin is loaded, the Framework ensures that its version is compatible with the Framework version that the plugin expects.

Plugins are dynamic libraries that provide one or more interfaces. An Interface is an API (with a versioned, stable ABI) that applications can use to access functionality in a plugin.

About Interfaces and Authoring a Plugin

See the Guide for a walkthrough of authoring a plugin, and information about interfaces and versioning.

Starting Carbonite

The Carbonite Framework is found in carb.dll (Windows), libcarb.so (Linux) or libcarb.dylib (Mac). You can either use the package-provided import library to import the functions from the Framework, or dynamically load and request the functions by name (rare).

There are two requirements for a Carbonite application: declaration of globals, and starting the Framework.

Declaration of Globals

The OMNI_APP_GLOBALS macro specifies the Carbonite application globals:

OMNI_APP_GLOBALS("example.windowing.native.app", "Native (C++) example app using example.windowing.");

This macro ensures that g_carbClientName is set to the Client name, and that the default logging channel is set up (the description provided is used as the description for the default logging channel).

Historically, this macro instantiated global variables such as g_carbFramework and g_carbLogging, but this was changed with Carbonite release v117.0 so that these variables are weakly-linked and need not be instantiated.

For backwards compatibility, older names of this macro exist: CARB_GLOBALS_EX (equivalent to OMNI_APP_GLOBALS) and CARB_GLOBALS (which does not allow specifying a description).

Starting the Framework

Starting the Framework is as easy as using the OMNI_CORE_INIT macro.

int main(int argc, char** argv)
{
    // Startup the Framework
    OMNI_CORE_INIT(argc, argv);

    carb::Framework* framework = carb::getFramework();
    if (!framework)
        return EXIT_FAILURE;

    return runApp(framework);
}

Typically, command-line arguments are passed to the OMNI_CORE_INIT macro, but other options are available (see documentation).

This macro ensures that the Framework is acquired (calling carb::acquireFramework()), the Omniverse Native Interfaces subsystem is started (calling omniCoreStart()), and finally calls carb::startupFramework() to initialize the Framework.

The carb::startupFramework() function can potentially do a large amount of work. Based on parameters it can load the settings plugin, parse command-line options and environment variables, and register plugins indicated in the configuration file. Check the documentation for carb::startupFramework() for more information.

Clients and Naming

Any entity which uses carb::Framework or is a plugin is a Client. This generally includes the Carbonite Application itself, as well as any plugins and other libraries (such as script bindings) that interact with the Carbonite Framework. All Clients must have unique Client Names as the Framework tracks usages and dependencies by these names. Carbonite Applications declare their Client Name in their Globals declaration. Plugins declare their Client Name in the carb::PluginImplDesc struct passed to the CARB_PLUGIN_IMPL macro. Script bindings declare their Client Name in their CARB_BINDINGS declaration.

If there are multiple definitions of the same interface, the general rule is to uniquely name each plugin by adding a qualifier after the interface name (e.g. carb.profiler-cpu.plugin, carb.profiler-tracy.plugin, etc.). This approach also means that you cannot have multiple implementations of the same interface in the same plugin (by design).

Finding and Registering Plugins

Plugins must be registered before they can be used. When a plugin is registered, the following occurs:

There are several different ways that plugins may be found and registered:

Registered when starting the Framework

As mentioned above, plugins can be registered by carb::startupFramework(), typically called from OMNI_CORE_INIT.

Parameters to carb::startupFramework() are passed in carb::StartupFrameworkDesc via OMNI_CORE_INIT.

If a configuration file or string is passed to carb::startupFramework(), the function will attempt to register the specified plugins. See carb::detail::loadPluginsFromConfig() for a list of the settings keys and how they’re used.

Searching for Plugins Programmatically

The carb::Framework::loadPlugins() function uses carb::PluginLoadingDesc to describe how to search directories to locate and load plugins. This allows searching directories for plugins by wildcard and even excluding plugins.

This is the most common means of finding and loading plugins after startup.

Explicitly Registering a Plugin

The carb::Framework::loadPlugin function allows loading a specific plugin, given the path. This option is rarely used.

Registering a Static Plugin

In rare cases, an application can register a plugin that actually exists in the application or a plugin. This is used occasionally when an application wants to have its own implementation of an interface, or to mock-up an interface. This is accomplished through the carb::Framework::registerPlugin function. Plugins registered via this method are never unloaded when their interfaces are released; instead they must be explicitly unloaded with carb::Framework::unregisterPlugin.

Loading Plugins

Typically when a plugin is found, it is loaded immediately. This means the dynamic library is loaded and static initializers within the library will run, and the Framework will locate and call functions exported by the plugin that the Framework uses to discover information about the plugin as described above.

However, if carb::PluginLoadingDesc::unloadPlugins is true, the plugin will be loaded to gather information, but immediately unloaded until an interface is requested. This is rare as it is not the default option. In some cases, the OS loader could choose to keep the library loaded, in which case a warning is logged.

Acquiring Interfaces

Acquiring an interface gives you an ABI-stable API that you can use to access plugin functionality. Once a plugin is registered and loaded, the Framework will be able to provide interfaces from it.

The first time an interface is acquired from a registered plugin, the plugin is started. To do this, the Framework calls carbOnPluginStartupEx() (or carbOnPluginStartup()). If startup succeeds, the Framework will provide interfaces from the plugin back to the caller.

getCachedInterface

This is the preferred means of acquiring an interface. Since carb::Framework::tryAcquireInterface does a lot of checking, it can be expensive when repeatedly called (plus, it produces log messages). In order to provide a very cheap way to cache an interface, carb::getCachedInterface() is provided. This is conceptually similar to:

// NOTE: Do not actually do this; use carb::getCachedInterface<carb::dictionary::IDictionary>() instead.
static auto dictionary = carb::getFramework()->tryAcquireInterface<carb::dictionary::IDictionary>();

However, unlike static, getCachedInterface() will reset its internal pointer to nullptr when the interface (or the Framework) has been released; calling getCachedInterface() after this point will attempt to re-acquire the interface.

The interface acquired will be from the default plugin. A specific plugin name can be provided from which to acquire the interface (but this has caveats). Given the template nature of carb::getCachedInterface(), specifying a plugin requires some special handling in the form of a global char array:

const char sGamma[] = "carb.frameworktest.gamma.plugin";
TEST_CASE("Specific getCachedInterface", "joshuakr", "[framework]")
{
    FrameworkScoped f;
    f.loadPlugins({ "carb.frameworktest.*.plugin" });

    auto iface = carb::getCachedInterface<carb::frameworktest::FrameworkTest, sGamma>();
    REQUIRE(iface);

    CHECK_EQ(iface->getName(), "gamma");
}

tryAcquireInterface

The most common method of acquiring an interface (and the method used by carb::getCachedInterface()) is carb::Framework::tryAcquireInterface:

// Acquire the IDictionary interface (typically this would be carb::getCachedInterface() instead):
carb::dictionary::IDictionary* dict = carb::getFramework()->tryAcquireInterface<carb::dictionary::IDictionary>();

If the interface is not available nullptr is returned. To debug why an interface cannot be acquired, use verbose logging.

The Framework provides the interface from the default plugin. A specific plugin name can be provided from which to acquire the interface (but this has caveats):

auto profiler = carb::getFramework()->tryAcquireInterface<carb::profiler::IProfiler>("carb.profiler-cpu.plugin");

Warning

As carb::Framework::tryAcquireInterface does some level of work for each call with potential logging, it is highly recommended to instead use carb::getCachedInterface() instead.

acquireInterface

Warning

This function should only be used within plugins and only for interfaces which are declared as dependencies (with CARB_PLUGIN_IMPL_DEPS). If it fails to acquire an interface, an error log is issued; this cannot happen for declared dependencies within a plugin as the plugin will fail to load if the dependencies cannot be acquired.

carb::Framework::acquireInterface is similar to carb::Framework::tryAcquireInterface, but will issue an error log message if the interface cannot be acquired. It will also warn if a Client requests an interface that is not declared as a dependency with CARB_PLUGIN_IMPL_DEPS.

Interfaces provided by the same plugin that is acquiring it will not result in any log or warning messages, and need not (nor can be) specified as a dependency.

Generally this method should be avoided unless it is guaranteed that the interface would be available, either by declared dependency or by also being provided by the same plugin.

The Framework provides the interface from the default plugin. A specific plugin name can be provided from which to acquire the interface (but this has caveats):

auto profiler = carb::getFramework()->acquireInterface<carb::profiler::IProfiler>("carb.profiler-cpu.plugin");

Because plugin dependencies can specify only interfaces, not plugins, it is recommended to use carb::Framework::tryAcquireInterface or carb::getCachedInterface() instead.

Acquiring an Interface from a Specific Plugin

carb::getCachedInterface(), carb::Framework::tryAcquireInterface and carb::Framework::acquireInterface all allow specifying the client name of a plugin (e.g. “carb.profiler-cpu.plugin”) to acquire the interface from a specific plugin.

In most cases, while multiple plugins may provide the same interface, there is nuance to usage. The Profiler example stands here as the profilers require external tools (carb.profiler-tracy.plugin or carb.profiler-nvtx.plugin), or in the case of the serializers (carb.dictionary.serializer-toml.plugin or carb.dictionary.serializer-json.plugin) read and write completely different file types. So while the contract specified by the interface definition is consistent, additional care and programming is generally necessary in using them.

The load order may be non-deterministic and seemingly random. It is generally considered good practice if multiple plugins provide a given interface to set a default plugin.

It is also possible to acquire an interface from the path to the dynamic library file itself. If the dynamic library is not a known plugin, it is registered as a new plugin and loaded. This is accomplished with the carb::Framework::tryAcquireInterfaceFromLibrary() function.

Acquiring an Interface from the Same Plugin as Another Interface

As a very rare case, it may be desirable to acquire an interface exported by the same plugin as a different interface. To accomplish this, alternate versions of carb::Framework::acquireInterface and carb::Framework::tryAcquireInterface exist that take any interface pointer.

Client Spoofing

This is very rarely needed.

Using internal functions, it is possible to “spoof” a plugin. That is, your plugin or application can instruct the Framework to acquire an interface as if it were a different plugin requesting. This is useful for modifying unload order by notifying the Framework of a dependency between two plugins:

    // acquire beta as if we are alpha
    auto beta2AsAlpha = static_cast<FrameworkTestSecond*>(f->acquireInterfaceWithClient(
        "carb.frameworktest.alpha.plugin", interfaceDesc2, "carb.frameworktest.beta.plugin"));
    CARB_UNUSED(beta2AsAlpha);

    // acquire gamma2 as if we are beta
    auto gamma2AsBeta = static_cast<FrameworkTestSecond*>(f->acquireInterfaceWithClient(
        "carb.frameworktest.beta.plugin", interfaceDesc2, "carb.frameworktest.gamma.plugin"));
    CARB_UNUSED(gamma2AsBeta);

    // acquire gamma as if we are alpha
    auto gammaAsAlpha = static_cast<FrameworkTest*>(f->acquireInterfaceWithClient(
        "carb.frameworktest.alpha.plugin", interfaceDesc, "carb.frameworktest.gamma.plugin"));
    CARB_UNUSED(gammaAsAlpha);

Try Acquiring an Interface Without Loading

This is very rarely needed. It may be needed in circumstances where a circular dependency exists between interfaces and the Framework chooses to shut down a dependent interface prior to plugin shutdown (carbOnPluginShutdown()).

All of the above options will make an attempt to load the plugin in order to acquire the interface. Sometimes this is undesirable– the interface should only be acquired if its plugin is already loaded and the interface acquired elsewhere. This can be accomplished via the carb::Framework::tryAcquireExistingInterface() function.

This is the case in the Profiler system where Settings is used only if it is already available. This is necessary because the Profiler could be loading and it is dangerous to try to load Settings recursively.

        // Don't try to load settings, but if it's already available we will load settings from it.
        auto settings = g_carbFramework->tryAcquireExistingInterface<settings::ISettings>();

Default Plugins

In some cases, it is reasonable to have multiple options for interfaces. A typical example is carb::profiler::IProfiler which is provided by multiple Carbonite plugins: carb.profiler-cpu.plugin, carb.profiler-tracy.plugin, and carb.profiler-nvtx.plugin.

There are multiple ways of declaring the default plugin.

Version

The implicit default plugin is the Framework’s earliest registered match for the highest compatible version for the requested interface. In order to be a compatible version, the major version of the plugin’s interface must match the major version of the requested interface, and the minor version of the plugin’s interface must be greater-than-or-equal to the minor version of the requested interface.

Configuration

The configuration file may have a /defaultPlugins key that specifies plugin names. See carb::detail::setDefaultPluginsFromConfig() for more information. Note that a listed plugin will become the default for all interfaces that it provides.

Programmatically

An application or plugin may specify a plugin to use as default with the carb::Framework::setDefaultPlugin() function.

For this function to be useful, it must be called to register a default plugin before the interface in question is acquired by any clients. This should be done as close to Framework startup as possible.

Overriding

As mentioned above, the carb::Framework::tryAcquireInterface and carb::Framework::acquireInterface functions allow explicitly providing a plugin name to acquire the interface from that plugin. This will ignore any default plugin.

Releasing Interfaces

When you acquired a particular interface, carb::Framework tracks this using client name. Usually you don’t need to explicitly release an interface you use. When the plugin is unloaded, all interfaces it acquired are released.

That being said, you can explicitly release with the carb::Framework::releaseInterface function when an interface is not needed anymore.

When an interface was acquired by a client this information is stored in carb::Framework. Multiple acquire calls do not add up. This means that:

IFoo* foo = framework->acquireInterface<IFoo>();
IFoo* foo2 = framework->acquireInterface<IFoo>();
IFoo* foo3 = framework->acquireInterface<IFoo>();
framework->releaseInterface(foo);

will release all foo, foo2, foo3, which actually all contain the same pointer value.

Remember that a plugin can implement multiple interfaces. Every interface can be used by multiple clients. Once all interfaces are released by all clients (explicitly or automatically) the plugin is unloaded.

Unloading Plugins

Once all interfaces to a plugin are released, the plugin is automatically unloaded. However, if a plugin never had any interfaces acquired from it, it remains loaded. It can be explicitly unloaded by plugin name with carb::Framework::unregisterPlugin or by library path with carb::Framework::unloadPlugin, but given the automatic unloading, this is typically not necessary for dynamic plugins.

Both of these functions also unregister the plugin, so attempting to acquire an interface from it again will not work without re-registering.

It is also possible to unload all plugins with carb::Framework::unloadAllPlugins. This is typically done automatically when the Framework is released.

Plugin Dependencies

Plugins may declare dependencies on interfaces that are required to be available before the plugin. An example of this is carb.settings.plugin which is dependent upon carb.dictionary.plugin. These are specified with CARB_PLUGIN_IMPL_DEPS generally immediately following the CARB_PLUGIN_IMPL use:

const struct carb::PluginImplDesc kPluginImpl = { "carb.settings.plugin", "Settings storage", "NVIDIA",
                                                  carb::PluginHotReload::eDisabled, "dev" };
CARB_PLUGIN_IMPL(kPluginImpl, carb::settings::ISettings)
CARB_PLUGIN_IMPL_DEPS(carb::dictionary::IDictionary)

In this case, when carb::getCachedInterface<carb::settings::ISettings>() executes for the first time, before initializing carb.settings.plugin, the Framework will first try to acquire carb::dictionary::IDictionary from an available plugin (typically carb.dictionary.plugin). Missing dependencies will cause a plugin load to fail, and nullptr will be returned as the requested interface could not be acquired.

Carbonite provides a command-line tool, plugin.inspector which, given a particular plugin library, can list which interfaces it depends on. Example:

plugin.inspector.exe carb.assets.plugin.dll

Its output:

{
    "carb.assets.plugin": {
        "name": "carb.assets.plugin",
        "path": "d:/Projects/Carbonite/_build/windows-x86_64/debug/carb.assets.plugin.dll",
        "description": "Assets.",
        "author": "NVIDIA",
        "build": "dev",
        "interfaces": [
            {
                "name": "carb::assets::IAssets",
                "major": 0,
                "minor": 2
            }
        ],
        "dependencies": [
            {
                "name": "carb::tasking::ITasking",
                "major": 0,
                "minor": 1
            },
            {
                "name": "carb::datasource::IDataSource",
                "major": 0,
                "minor": 3
            }
        ]
    }
}

Listing an interface as a dependency allows your plugin to use carb::Framework::acquireInterface (as opposed to carb::Framework::tryAcquireInterface) without any fear of error logging or nullptr returns.

A best practice is to only list interfaces as dependencies if your plugin cannot function without them. Other interfaces can be considered optional and your plugin can handle the case where the interface is not available. An example of this is carb.settings.plugin which requires carb::dictionary::IDictionary (and lists it as a dependency as seen above), but carb.profiler-cpu.plugin can function just fine without carb::settings::ISettings; it merely applies default settings. In this case, carb::settings::ISettings is not listed as a dependency in CARB_PLUGIN_IMPL_DEPS but the plugin uses carb::Framework::tryAcquireInterface and handles the case where nullptr is returned.

It’s important to note that dependencies are per plugin and set by the plugin author. Different plugins might implement the same interface, but have different dependencies.

Circular explicit dependencies are not allowed. Having two plugins that are dependent on interfaces from each other will result in an error log message and nullptr return from carb::Framework::acquireInterface. However, it is perfectly reasonable to allow circular use of interfaces that are acquired optionally upon first use. In this case, do not list the interface in CARB_PLUGIN_IMPL_DEPS and use carb::Framework::tryAcquireInterface when needed, sometime after startup (i.e. when carbOnPluginStartup() is called).

Unload Ordering

CARB_PLUGIN_IMPL_DEPS specifies explicit dependencies, but any use of carb::Framework::acquireInterface or carb::Framework::tryAcquireInterface (or any of the variations) notifies the Framework of an implicit dependency. The Framework uses this information to determine the order in which plugins should be unloaded.

In general, the Framework will try to unload dependencies after the plugin in which they’re required. In other words, given our example of carb.settings.plugin dependent on carb::dictionary::IDictionary (typically from carb.dictionary.plugin), the Framework would attempt to unload carb.settings.plugin before the plugin providing IDictionary. This allows carb.settings.plugin to continue using IDictionary until it is fully shut down, at which point IDictionary can be shut down.

Since circular usage is allowed (and therefore, circular implicit dependencies), the Framework may not be able to satisfy the above rule. In this case, the Framework will unload in reverse of load order (unload most recently loaded first).

Unload ordering is very complicated and requires that the Framework have an understanding of the explicit and implicit dependencies of all loading plugins. Because acquiring an interface is an implicit dependency, whenever an interface is acquired the Framework will re-evaluate the unload order and log it out (Verbose log level). Turning on Verbose logging can help diagnose unload ordering issues.

Versioning

The Carbonite Framework uses the concept of semantic versioning to determine a plugin’s compatibility with the Framework, and whether an interface provided by a plugin is compatible with an interface request. Carbonite uses major and minor version numbers to do this:

  • The major version number of the candidate must match the major version of the request.

  • The minor version number of the candidate must be greater-than-or-equal to the minor version of the request.

Carbonite expresses its concept of a version with the carb::Version struct and checking with comparison operators and the carb::isVersionSemanticallyCompatible() function.

Framework Compatibility

When a plugin is registered with the Framework, the Framework will find and execute the carbGetFrameworkVersion() function exported by the plugin. This is the version of the Framework that the plugin was built against. In this case, the plugin is considered the requestor and the Framework is considered the candidate. Therefore, the Framework minor version must be greater-than-or-equal the minor version returned by carbGetFrameworkVersion() (and the major versions must match exactly).

If the Framework determines that the plugin is not compatible with it, it will refuse to load.

In order to support plugins compiled at different times, the Carbonite dynamic library will honor multiple versions of the Carbonite Framework. The latest Carbonite version is available as carb::kFrameworkVersion.

Interface Compatibility

For the many variations on acquiring an interface, a version check is also performed. The Client that calls carb::Framework::tryAcquireInterface (for example) is considered the requestor. The plugins that have been registered with the Framework provide interfaces that are considered the candidates. Every interface has a InterfaceType::getInterfaceDesc() function that is generated by the CARB_PLUGIN_INTERFACE or CARB_PLUGIN_INTERFACE_EX macros (example function: carb::tasking::ITasking::getInterfaceDesc()).

The carb::Framework::tryAcquireInterface (for example) is a templated inline function that takes the interface type as its template parameter T and will call InterfaceType::getInterfaceDesc() to provide the interface name and semantic version to the Framework at interface acquire time. When a plugin or application is built, this inline function is written into a compilation unit and captures the name and version that the plugin or application knew about when it was built. This information forms the request.

At runtime, the plugin or application passes the interface name and version information that it was built with. If a specific plugin was given, the Framework checks only that plugin; otherwise the default plugin is used (if no default plugin has been specified or identified, the Framework will select the first registered plugin — in registration order — with a greater-than-or-equal version to the interface version requested). From this plugin, the Framework will check to see if an interface candidate is exported whose major version matches exactly and whose minor version is greater-than-or-equal to the requested minor version.

If the version of the candidate has a higher major version, the Framework will ask the plugin if it can create an interface for the requested version. This only happens if the plugin supports multiple interface versions.

If no matches are found, nullptr is returned to the caller.